We can broaden the scope and thus the utility of our paper beyond breast cancer, e.g. skin cancers and whatever is detectable by local change of heat generation over time through the modern days  convenient and powerful multiple spectral image processing deep learning (DL) algorithm by Smartphone’s. We have proposed a novel Smartphone day & dual IR night (e.g. from the day visible – 0.4 – 0.75 microns - and to the night near infrared – 0.78 – 3 microns – VISNIR, medium wave infrared - 3-5 microns – MW and long wave infrared - 8-12 microns – LW).  Artificial Neural Network emulate Human Visual System Cortex 17 layer by layer so-called Deep Learning Algorithm loaded in Smartphone which may be able to spot early and advanced signs of tumors formations in the all skin related region. Furthermore, we adopt the available data from Nat. Lib. of Medicine (loc. National Institute of Health, cf. Fig.1 ) data-guided semi-unsupervised learning at our algorithm at min. False Negative Rate).  As long as we do not change the cancer diagnosis standard, e.g. taking samples and growing on a Petri dish, etc. and letting the physicians make the final decision, we do not have legal liability, but moral obligations.

The significance of the proposed technique relies on three facts as follows:

1. Since the number of male about 3000 annually in the USA alone and growing, they tend not to have protruded but flat Therefore, they do not easily impose the horizontal and vertical compression into so-to-speak “pancake shapes” for X-ray mammogram to detect a later stage of micro-calcification. Dual IR Smartphone imaging computing can broaden early warning from breast to skin for early stage of any anomaly growth rates.
2. The much lower tumor localization specificity and sensitivity obtained using just one spectral band based on the Planck blackbody radiation curves, one per temperature.
3. The better statistical analysis obtainable by our home-based method.
4. In some detail, although a single Infrared (IR) camera can already detect a hot spot emitted from a warm human body at in a ambient controlled room $(\because 37°C\cong \left(\frac{1}{37}\right)eV)$ in cases of hidden and minute amounts of temperature increases  this method might become detrimental; since its small temperature variations’ lower detection rate the so-called False Negative Rate would almost certainly go higher causing also the inadvertent and possible delay in the diagnosis and localization of cancer independently of its stage; namely anywhere from:

1. Stage #1: Inception to
2. Stage #2: Angiogenesis or even
3. Stage #3: formation by Anaerobic tumor to the
4. Stage #4: Extrications cancer gene transporting through lymph nodes to any other body part. [more details of cancer stages will follow]

In this review, we propose to use a widely available Smartphone augmented with two distinct infrared (IR) spectrum filters [commercial available], not one, passive acquisitions, e.g.

1. Middle Wave (MW) IR and
2. Long Wave (LW) IR.

Furthermore, we take the advantage of widely available annual budget of $31B resource of 30 National Institutes of Health (NIH) and one National Library of Medicine (NLM) as follows:

Figure 1 Smartphone empowered by modern communication:  (a) Modern Internet Clound Computing: The communication frequency will be above 10 GHz.  The rule of thumb is that one GHz is one big foot about 30 cm. Then, above 10 GHz will be shorter than 3 cm into mm wave regime. According to the Nyquist sampling theory, “a band-limited continuous-time signal can be sampled and reconstructed from its discrete samples, if it is over twice as fast as its highest frequency component.” In fact, the download speed will be about 1 giga bytes per second (gbps). The size of antenna is tremendously reduced at m.m.w., the cost of antenna tower is significantly lower as well and easily mounted, e.g. on a lamp post.  Of course, the caveat is the m.m.w. penetration will be significantly reduced and the cell power density will have to increase. That said, the gain for BMW is tremendous,; (b) The difference of traditional 4-th Generation cellular phone towers versus 5-th Generation (5G) millimeter wave (m.m.w.) cellular towers. Note that New 5G requires a lot more towers say lamp-posts, window frames, due to the lack of propagation and penetration depth. It has been implemented in the most of Asian Countries and also in the US Cities by the next April 2020, because of the success of test beds, e.g. Verizon in Chicago.   Archival Medical Data Storage (c) NIH/Nat Lib Med, (d) Personal tours and collaborations are possible.  Thus, we can send simultaneously home care surveillance data analysis, relay to personnel physician(s) and/or 911 centers. Empowered Smartphone’s: (e) Smartphone can have both day & night imaging with filter Filters usually range from 590 to 800-900nm and usually, this kind of colored infrared shots are obtained with 590nm on a model camera because it lets some visible & IR through.

Figure 1 A single infrared camera took two images of a healthy woman in a room cooled to 21o C. Color indicates temperature. After 10 minutes, the image on the right shows most heat has dissipated, and no abnormal heat emission within either breast is visible.

Our body cellular heat generation and acquisitions cf. Figure 1 (e,f) are embodied by two (LW and MW) spectral filters augmented at the camera lens of a Smartphone camera, mounted on a fixed point triples in a dark room to achieve stable multispectral imaging (VISNIR + MW + LW); e.g. the Smartphone camera can be supplemented by two screwed-on IR filter lens  (or sophisticated with built–in optical beam splitter for LW & MW), with the focal plane registered at any of 3 fixed body points (arm pits including 2 nipples).  The acquisition trade-off procedure is clear with some trials and errors benchmarked with the NIH/NCI Gold Standard including X-ray Micro calcification & NLM archival dissection of man and woman in digital layers and other medical image data. Then one can perform layman Smartphone augmentation in a normal house room (temperature, light etc. registered by Smartphone powerful computer within) or under the Smartphone interface guidance to find a place approximating a temperature and light controlled ambient where the patients can sit alone for the IR acquisitions. In each acquisition skin type the patients have to disrobe their upper body, cool down (~15^oC) temporarily their hands or the upper torso (to lower the S/N ratio) and acquire two snapshots in two consecutive days. The Smartphone software will then determine the temperature of the acquired images according to the Planck blackbody radiation curves, and keep the T(x, y, t) image pixel values and compute their temporal differences $\Delta T\left(x,y,t+1\right)-\Delta T\left(x,y,t\right)$ , to label each pixel with a binary classification:  In fact, we adopt pair of eye with a pair of daylight and dual IR night-light Smartphone’s algorithm, we can reject any fluctuations in household environments.

• “1, agree, the signal”;
• “0, disagree, the background noise”.

Mathematical model & semi-supervised smart deep learning algorithm

The math of semi-unsupervised deep learning would be useful for computing the tradeoff of minimize the False Negative Rate (FNR) after many tests & evaluations.

Before we choose a Gaussian statistics to describe our experiments, we shall mention another choice, e.g. the Levy statistics with its specific Cauchy density.  French mathematician Cauchy kept the first order Taylor expansion of the Gaussian density and re-normalized it as Cauchy density:

$\exp (-x^2)\cong 1-x^2\cong \frac{1}{1+x^2}$

${\rho }_C\left(x\right)=\frac{1}{\pi }\frac{1}{1+x^2}$

which cannot be reduced to the Gaussian one, because < x 2 >=∞ MathType@MTEF@5@5@+= feaagKart1ev2aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqkY=Mj0xXdbba91rFfpec8Eeeu0xXdbba9frFj0=OqFf ea0dXdd9vqaq=JfrVkFHe9pgea0dXdar=Jb9hs0dXdbPYxe9vr0=vr 0=vqpWqaaeaabiGaciaacaqabeaadaqaaqaaaOqaaiabgYda8iaadI hadaahaaWcbeqaaiaaikdaaaGccqGH+aGpcqGH9aqpcqGHEisPaaa@3D81@   For simplicity, in this preliminary study, we shall not to choose the fat tail Cauchy ${\rho }_C\left(x\right)$  distribution to describe the large number cancers cases.

Innovation: this Gaussian probability extension is intended as a test bed that works as an approximation constructed around the center of all quasi-Gaussians distributions. We wish to demonstrate how to save the cost of specialized medical operators from a purely supervised learning that requires the acquisition of huge datasets. Nonetheless, any False Negative Rate (FNR) must be identified and eliminated by utilizing the Bayesian conditional probability to lower the FNR. This approach is commonly known as semi-supervised learning.

Approach: In order to save Medicare cost, and oncologist time, by building a psychological mirror, we wish to extend the actual data set $\left\{x\right\}\subseteq \left\{BDA\right\}$  via a Gaussian-extension  by using randomly selected examples out of a Big Data Analysis (BDA) database to assign the discrete cancerous categories vector[t+t’+’t ̃], where supervised data set  are assigned by the costly oncologists.

We consider a simpler application of chest heat emission for breast cancers, or specifically Ductile Carcinoma In Situ (DCIS) where man has about 3000 per year.  Furthermore, in the US alone due to the Nulliparous [10] Fig. 2 Appendix and/or overweight may amplify each other's effect on breast cancer risk among women after menopause toward golden age. 

LMS: $E={\left|t-x\right|}^2\cong 0$     (1)

The trick is assuming the normal distribution peak around randomly selected exemplars, and taking the Gaussian ensemble average in angular bracket of the supervised training in the Least (Mean) Square error (LMS) Energy sense as follows:

Ensemble averaged LMS

$t\subseteq t^{″}\equiv t+t^{\prime }+{\tilde{t}}^{\prime }x\subseteq x^{\prime }\equiv x+x^{\prime }$   (2)

$\overline{E}=<{\left|t^{″}-x^{\prime }\right|}^2>G\left(t^{″}\right)G\left(x^{\prime }\right)={\displaystyle \int d}{x}^{\prime }\exp \left(-{\left|x^{\prime }\right|}^2\right){\displaystyle \int \exp \left(-{\left|t^{″}\right|}^2\right)}|t^{″}-{x^{\prime }|}^{2}d{t}^{″}$   (3)

Smartphone hardware existed; the software classifier might be programmed as semi-supervised deep learning minimized the FNR as follows:

The machine had extended to a much larger class $t^{″}$ , and required a larger training data $x^{″}$ which might be smuggling in False Negative Rate cases (FNR) inadvertently in both samples delaying the patient’s precious time for early treatment (while False Positive Rate (FPR) is merely a nuisance also if with an associated economical cost). 

${\sigma }_{FAR}\left(t^{″},x^{\prime }\right)={\sigma }_{FPR}\left(t^{″},x^{\prime }\right)+{\sigma }_{FNR}\left(t^{″},x^{\prime }\right)$   (4)

It’s preferably to have an extra gain without oncologist $t^{\prime }\gg {\tilde{t}}^{\prime }$ of which the FNR ${\tilde{t}}^{\prime }$   must be reduced. 

Figure 2 Gaussian extension and minimize the False Negative Rate side wing.  “Malignant cancer” super-Gaussian is denoted in Bold Face Dasher line  with the 4 sub-Gaussian with variance ${\sigma }_n$ ;n=1,2,3,4, …corresponding to (1) inception;(2) growth with new blood supply line neo-angiogenesis;(3) tumor energy anaerobic production so-called Warburg effect;(4) the metastasis-exportation. It might be clear that ${\sigma }_{FNR}$  located schematically in the early inception stage of cancers (LHS), rather the later stages (RHS). We wish to generalize 4 cases into one large Gaussian category (in bold dashes).  Then, more unidentified malignant and benign cases will be indicated in dashes.

Figure 2 A single infrared camera took images of a breast cancer patient in a room cooled to 21o C. After 10 minutes, the image on the right shows active cancer cells (Angiogenesis) emitting abnormal heat, while the surrounding tissue has cooled.

BDA experimental data should be generated beyond traditional micro-calcification by X-ray Mammograms, or oxygen-utilization with functional f-MRI, early stage tracking by non-invasive “Dual IR thermo-grams,”(by Szu et.al.)  are useful supplement for frequent private checkup for early detection of breast cancers,^1-10 [ e.g. Nulliparous women coined for  Singapore epidemiology studies for Singapore women Figure 1 having 5 children before WWII, and ~1 after the War; passed to Vatican Cardinal Camillo about the potential risk of nuns of the Vatican, and elsewhere [ cf. “Breast Cancer in Singapore: Trends in Incidence 1968-1992,” A. SEOW, S. W. DUFFY, M. A. McGEE, J. LEE & H. P. LEE, Int’l J. Epidemiology, Vol. 25, No. 1, pp.40-45 (1996 Britain)].

Figure 3 Falling fertility on increasing breast cancer risk taken from I’tl  J. 10. “Breast Cancer in Singapore: Trends in Incidence 1968-1992,” A SEOW, S W DUFFY, M A McGEE, J LEE & H P LEE, Int’l J. Epidemiology, Vol. 25, No. 1, pp.40-45 (1996 Britain). H.S. received an embroider mural from Pope John Paul II:”Bless Harold And His Family (unfortunately lost due to shipment packages).

Figure 3 (Double-Blind Test Result): Two-camera Multispectral Infrared breast image. Left figure is medium wavelength IR and right figure is a long wavelength IR breast imagery. The neural network unsupervised classification algorithm can do detection from one set of imagery without a library.

Some Unsupervised Learning at the Min. Free Energy to reduce the risk of FNR:  Derivation of Newtonian-like equation of motion from the Lyaponov monotonic convergence. Since we know $\Delta \overline{E}\le 0$

Lyaponov:  $\frac{\Delta \overline{E}}{\Delta t}=\left(\frac{\Delta \overline{E}}{\Delta \left[W_{i,j}\right]}\right)\frac{\Delta \left[W_{i,j}\right]}{\Delta t}=-\frac{\Delta \left[W_{i,j}\right]}{\Delta t}\frac{\Delta \left[W_{i,j}\right]}{\Delta t}=-{\left(\frac{\Delta \left[W_{i,j}\right]}{\Delta t}\right)}^2\le 0$  Q.E.D.      (5)

Therefore, the Newtonian equation of motion for the learning of synaptic weight matrix $\left[W_{i,j}\right]$ follows from the equilibrium at a minimum energy

Newtonian-like:  $\frac{\Delta \left[W_{i,j}\right]}{\Delta t}=-\frac{\Delta \overline{E}}{\Delta \left[W_{i,j}\right]}$     (6)

In the deep layer learning (taken from¹“Deep learning,” Yann LeCun, Yoshua Bengio & Geoffrey Hinton, Nature 521, 436–444 (28 May 2015). While the input data flows from the bottom to the top, the cost function E reduces from the top to bottom.  Once upon a time, Academician David Remelhart of MIT PDP books 1984 gave the most succinct talk about “backward error propagation, “in a lecture. He said “when your hands touch the hot stove, your hands blame backward your control muscle, your control muscle, blame your eyes sensing. (change it to your hands blame your control muscle which in turn blames your eyesight” We compare outputs with the desirable target t called error derivative to propagate backward to hidden k-th layer (hand) and to j-layer (control) the i-th eyes layer and then to input layer xi..   The l-th layer output is compared with desired target t, the equations used for computing the backward pass. At each hidden layer we compute the error derivative with respect to the output of each unit, which is a weighted sum of the error derivatives with respect to the total inputs to the units in the layer above. We then convert the error derivative with respect to the output into the error derivative with respect to the input by multiplying it by the gradient of (𝑧). At the output layer, the error derivative with respect to the output of a unit is computed by differentiating the cost function which gives $2\left(x_i-t^{″}\right)$ if the cost function for unit ${\left|x_i-t^{″}\right|}^2$ , where 𝑡” is the output l-th layer with a larger target value defined by Eq(3).

The target goal is denoted by the double-primed letter $t^{″}$ , while the input data as $“x”$ entering i layer to j layer via k layer to the output l layer.

$\frac{\partial E}{\partial x_k}=\left(t-x_k\right)$ ; $\frac{\partial E}{\partial x_k}={\displaystyle {\sum }_{k,lout}\left[w_{k,l}\right]}\frac{\partial E}{\partial z_l}$ ;

where  $\frac{\partial E}{\partial z_l}=\frac{\partial E}{\partial x_k}\frac{\partial x_k}{\partial z_l}$

and the weight is updated $\left[w_{k,l}\right]=w_{k,l}^o+\delta w_{k,l}$

Figure 4 cf. “Deep Learning,” Yann LeCun, Yoshua Bengio & Geoffrey Hinton, Nature 521,pp. 436–444 (28 May 2015) Algorithm and architecture. 

Figure 4 (Double-Blind Test Result): Unsupervised classification images of the left breast. Here the color scheme indicates the probability of normal heat source versus the abnormal heat source class.

Moreover, besides the DCIS hidden inside the milky ducts, the other carcinoma spots might be more difficult to detect due to the Fourier heat transport equation, the inside-out fan-out from warm blood in the old capillary or the new growth of capillary to pinpoint the location. Thus, we have sketched a pair of cameras to be conveniently augmented with a software-aided algorithm through a Smartphone camera (using mobile beam split filters attached lens).

1. Experimental design:

All these trials over 100 menopause women must be experimentally benchmarked by a radiologist using the traditional X-ray mammograms in subsequent collaboration. Reproductive History and Cancer Risk - National Cancer Institute seems to support the common sense “Used it or lose it” related to Epidemiology studies of breast cancer and the worse cases of nulliparous effect.

Biomedical engineering requires the design of Smartphone software and extension IR filters, software computation of Planck blackbody radiation and AI software to reduce the radiologist’s expenditure from 10 home dual IR and 1 hospital mammogram.  While the False Positive Rate (FPR) is a nuisance for patients to come back to check again, the False Negative Rate (FNR) is detrimental as it delays the time critical patients’ treatment for cancer.   

This multispectral setup potentially offers a passive, inexpensive, non-intrusive, convenient means of screening pre-cancer patients without radiation hazard, and may potentially detect carcinomas in situations long before a mammogram might detect them thanks also to the much higher frequency of acquisitions compared to the X-rays based ones.

Thermal breast scanning has been employed by DARPA for Tank to Tumor for a number of years; the technology has been even transferred to Europe (the Vatican) and Asia (NTU, Taiwan), but most Asian applications have been limited to a single infra-red band by acquisitions with a single camera. The application of the proposed Smartphone based MW and LW bands acquisitions and the semi-supervised classification algorithm may offer an unbiased, more sensitive, accurate, and generally more effective way to track the development of breast cancer, without the demanding conditions like a long wait in a cold, dark room and the associated variability and inaccuracy in thermal detection sometimes accompanied even by patient discomfort.

The success of the initial double-blind experiment substantiates the promising application for the use of combined MW and LW imaging in improving the early detection process for breast cancer and possibly other dermal carcinomas.

Moreover, we have already enormous reference cancers data for the background aging WWII baby boomers and correlation data. The Archival Big Data Analysis (archival-BDA) of cancers is available from 2 National Institutes Health (NIH) (Cancer Treatment and Cancer Prevention operated at an annual budget of $2B, of which a half goes to extra-mural funding). Furthermore, there is plenty of archival data available through the Internet and Cloud available at the Library of Medicine, on Wisconsin Ave, Bethesda next to the NIH Figure 1, where case-studies BDA can serve as the semi-supervised teachers.

With the advent of smart medical sensors, e.g. improved diabetic droplet sensor can measure other salient cancer stage data, which can further gather and communicate rapidly with 5G Wi-Fi channel for those home alone seniors (HAS).  Thus, we can gather real-time (real-time-BDA) supervised with the background of archival-BDA.

The Surge of Cancers might be due to the environment factor related to (e.g. stress & over-weight among inappropriate aging population (cf. Aging Population, Unhealthy Habits Underlie Expected Cancer Surge, Robert Preidt, Health-Day Reporter Oct. 16, 2019)

In this paper we review succinctly an old app of “early detection of cancers,” due to aging, and stress using the synergism between Artificial Intelligence (AI) and Big Data Analysis (BDA) to combine both in the so-called “Cutting Edge Artificial Intelligence (CEAI)”.  

Our review ended up with a big picture: namely recent Nobel Laureates, followed with a researcher and writer as well as well-known celebrities.

Figure 5 (1) 2019 Nobel Prizes have been given to Petter Ratcliffe(Oxford) , and Gregg L. Semanza (JHU) and William G. Kaelin, Jr. (Havard)  “for their discoveries of how cells sense and adapt to oxygen availability.” In other words, the detection of oxygen utilization to stimulate the grow of red blood cells in bone mallow that help control the metabolism under less stress improving the immune system, controlling the tumors growth; (2) 2018 Nobel Prizes have been given to Tasuku Honjo (Kyoto) and James P. Allison (U. Texas), James P. Allison recognizing for immune system T-cells and Tasuku Honjo identified cell programming death protein PD-1.  They have developed a game changer in cancer treatment ---therapies that work by harnessing the body’s own immune system by means of identifying cell programming death protein PD-1, as well as a type of immune white blood cells T cells.

Figure 5 (Double-Blind Test Result) Unsupervised classification images of the right breast. The left image shows normal heat distribution generated by the blood vessels. The right image indicates abnormal heat distribution as shown by the cyan circled area near the breast nipple. According to the doctor the diagnosis is Ductal Carcinoma In Situ (DCIS).

An important physiology fact is based on the stress affecting the hypothalamus leading to the pituitary gland connected to the adrenal cortex secreting Glucocorticoids that weaken the immune system.  This might be a “point of no return” between aging weakening defense epigenetic system breeding cancers. 

In general there are 4 stages of cancers:

1. Inception, the apoptosis (self-programming death) gene has been knocked out from our cells borne with genome, or life style epi-genome controlled by DNA Methylation (CH₃) silencing our cells. This is the most ambiguous stage of senescence or zombie-like cells. Any broadening of this stage without oncology in-situ culture might lead to False Alarm Rates (FAR).
2. Angiogenesis, new blood capillary vessels must be built to bypass micro-calcification to feed cancer cells growth;
3. Formation by Anaerobic tumors: In 1920s, Otto Warburg and colleagues made the observation that tumors were taking up enormous amounts of glucose was fermented to produce lactate, thus the term aerobic glycolysis; That’s how the solid mass tumor is being formed. This might manifest in the Warburg stage when cells use anaerobic production energy (e.g. similar to Fungi or yeast fermentation which breaks the sugar down to wine; in our case cancer cells become “cannibal”).  Thus, the cells can pileup packaging closely without the need of new blood vessel or neo-angiogenesis to supply the nutrients and oxygen.
4. Extrications cancer gene transporting through lymph nodes to all the other places. Specifically, breast cancer occurs when malignant tumors develop in the breast.  These cells can spread by breaking away from the original tumor and entering blood vessels or lymph vessels, which branch into tissues throughout the body. When cancer cells travel to other parts of the body and begin damaging other tissues and organs, the process is called the final stage of Metastasis.

The challenge is that cancer and normal tissues are macroscopically similar as pointed out in 2019 by David A. Sinclair (Harvard) in his book “Lifespan—Why we age, Why we Don’t have to”.  While President Jimmy Carter was cured from a brain tumor and now lives at age 96; Senator John McCain died of brain tumor at age 86.This indicates that not all tumors and immune capabilities are the same; for example, the cancer treatment drug by Merck called Keytruda (cf. in the following figure Yellow Color Balls). The product of two uncertainties makes the truth table of 4 entries. 

Figure 6 The Left Hand Side (LHS) shows our co-evolutional partner called the Mitochondria with their own simpler ring shape DNA, that can produce efficient energy power house within our own cells.  Without these dozen efficient Mitochondria power energy (imported during primordial evolution our symbiotic partners), our cells become senescence zombie-like cells which are macroscopically no difference to one’s own healthy cells, except worsening over time, Cancer cells might be developed during our weak immunological states. Right Hand Side (RHS) is taken from Sciencelearn.org.nz indicating the “genome” of double helix chromosome discovered by Watson and Cricket, and the environmental “phenomen” folding controlled by Cancer cells have macroscopically no difference to one’s own healthy cells, except worsening over time. It seems to begin by knocking out the Apoptosis gene (for self programming to death genome), or Epigenomic.(outside  Genome) by means of Histone protein and  Methynation () (for explicit expression of genome).  We quote: “If the Genome were a computer, (4 bits A,T,C,G), the Epigenome would be the software.” in an aforementioned book Sinclair, & LaPlante, review 2019, p.21.  Recently Frances M. Arnold, Caltech, et.al. had analyzed “Machine-learning-guided directed evolution with Gaussian perturbation for protein engineering, (Nature Methods. Vol.  16, Aug. 2019. pp. 687-694). The problem of finding optimum sequence is NP-hard, as no polynomial time to combine, e.g. 300 amino acids into 5,700 possible single amino acid substitutions (loc cit N.M.V. 16, p.689. They showed the uncertainty using Gaussian noise can help multiple layer Artificial Neural Nets deep learning for choosing sequence protein.

We aknowledge partial support of ONR Grant N00014 20 1 2279.

Office of

Naval Research

Press Release

800 N. Quincy St., Arlington, VA 22217-5660

http://www.onr.navy.mil

Detecting breast cancer with a new algorithm and a Multi-spectral Infra-Red imaging system

What does remote sensing for camouflaged enemy ground vehicles have to do with breast cancer diagnosis? By next year, perhaps plenty. A smart sensor fusion algorithm modeled on the human visual/brain “unsupervised” learning system and a 200 channel hyper spectral remote sensing capability have been developed by the Office of Naval Research for use as a passive electro-optical, infra-red ground surveillance system. The same method has now shown success in detecting the heat radiated by abnormally reproducing breast cancer cells.

Hyper spectral sensors sweep up enormous quantities of data, but their usefulness has been limited by our ability to pull the important information out of that clutter. The algorithm that processes the data is the important factor.   Last year the Under Secretary of Defense for Science and Technology asked ONR to look at the potential usage of remote sensing to improve breast cancer diagnosis. Dr. Harold Szu and Mr. James Buss’ single-pixel unsupervised classification algorithm, based on the Lagrange Constraint Neural Network (LCNN) and multiple spectral data per pixel initially designed to increase the effectiveness of surveillance systems, now promises to enhance the sensitivity and accuracy of breast cancer testing.

Abnormally reproducing cells demand greater nutrition through increased blood supply, thus generating higher concentrations of heat in specific areas. Applying their algorithm, Dr. Harold Szu and Mr. James Buss are able to classify the infrared heat distribution given off by these cells.

A truly unsupervised algorithm per pixel must be based on the information derived directly from spectral data alone. In order to reveal the hidden spectral features contained in a single pixel image data vector X=[A]S, one has to invert the matrix without knowing both the breast-medium heat-transfer matrix (MTF) [A] and the heat source S which both vary from pixel to pixel. While ONR's space-variant imaging algorithm following the spectral data vector analysis and the physics constraints of thermodynamics free energy minimization has achieved sub-pixel accuracy, other statistical Independent Component Analyses (ICA) methodologies suffer pixel-averaging blurring effect. This is because the average over neighborhood pixels must implicitly assume an identical MTF [A] for the space-invariant imaging. This would be true only in cases of a large tumor requiring no more automatic target detection.

Similar to a pair of human eyes, a pair of cameras – at different infrared wavelengths* – transcribes this thermal diffusion process into two images, which are then filtered for shared signals while disagreement noise is minimized. Through this process, last February Szu and Buss and their team detected early stage ductal carcinoma in situ (DCIS) in a test patient using a double-blind procedure.

“This multispectral, sub-pixel super-resolution is potentially more accurate by an order of magnitude,” states Dr. Szu, “It offers a passive, inexpensive, non-intrusive, convenient means of screening pre-cancer patients without radiation hazard, and may potentially detect in situ carcinomas long before a mammogram might detect them.”

Thermal breast scanning has been employed for a number of years, especially in Europe and Asia, but its use has been limited to a single infra-red band, using a single camera. The application of the “unsupervised” classification algorithm may offer an unbiased, more sensitive, accurate, and generally more effective way to track the development of breast cancer, without demanding the variables of a long wait in a cold room, increasing the variability inaccuracy in thermal detection and causing patient discomfort.

The success of the initial double-blind experiment substantiates the promising application for the use of multispectral imaging in improving the early detection process for breast cancer and possibly other dermal carcinomas. A provisional patent application has been filed. Follow-on research and clinical studies are being planned through the use of Cooperative Research and Development Agreements (CRADA). A web-based database of medical images (MedATR) is being developed by Advanced Concepts Analysis Inc., of Falls Church, VA, hosted on the Air Force Virtual Distributed Laboratory secure web site (VDL).

###

*Medium wavelength IR (3-5 m) camera and Long wavelength IR (8-12 m) camera. Both have about 10 milli Kelvin degrees in the minimum resolvable temperature difference (MRTD).

Released 4 Sept 02

For more information on this story, or to interview Dr. Szu and Dr. Buss, please call: Gail Cleere, 703-696-4987, or email cleereg@onr.navy.mil or Jennifer Huergo, 703-696-0950, or email huergoj@onr.navy.mil

1. Yann LeCun, Yoshua Bengio & Geoffrey Hinton, “Deep learning,” Nature. 2000;521:436–444.
2. Szu, H, Miao, L. Qi, H., “Thermodynamic free-energy minimization for unsupervised fusion of dual-color infrared breast images, Proc. SPIE Independent Component Analyses, Wavelets, Unsupervised Smart Sensors, and Neural Networks, Vol. 6247, April 2006;  Lidan Miao, Hairong Qi, Harold Szu: A Maximum Entropy Approach to Unsupervised Mixed-PixelDecomposition. IEEE Transactions on Image Processing. 2006;16(4):1008-1021.Lidan Miao, Hairong Qi, Harold Szu: Unsupervised Decomposition of Mixed Pixels Using the Maximum Entropy Principle. ICPR (1) 2006:1067‒1070.
3. Harold Szu, Philip Hoekstra, Joseph Landa, et al.”Neo-angiogenesis metabolic biomarker of tumor-genesis tracking by infrared joystick contact imaging in personalized homecare system,” SPIE Sensing Technology& Applications, 2014.
4. Artificial neural networks for noisy image super-resolution, Harold Szu, Ivica Kopriva. 2001.
5. Kenneth Byrd, Harold Szu, “Design of a cylindrical fiber-optic lens focusing passive dual-color IR spectra and readout,” 2007.
6. Edna Maria, Vissoci Reiche, Sandra Odebrecht, et al.“Stress, depression, the immune system, and cancer,” Oncology (Lancet, Elsevier). 2019;5:625‒634.
7. “Deep Larning,” Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature. 2015;521:436–444.
8. Ning Xi, Harold Szu, Jim Buss, Ingham Mack, et al.“Carbon Nanotube Mid Infrared Detectors Overlaid on Long Infrared Focal Plane Array,” SPIE, 2005, April for those who are sophisticated.
9. ONR-PR Detecting-Breast-Cancer-with-a-New-Dual-IR-Algorithm Cf. Appendix as follows.
10. A SEOW, S W DUFFY, M A McGEE, et al. “Breast Cancer in Singapore: Trends in Incidence 1968-1992,” Int’l J Epidemiology. 2000;25(1):40‒45.
11. Chia-Yen Lee, Hsin-Yu Hsieh, Si-Chen Lee, et al. Spatiotemporal sharpening of sub-pixel super-resolution by means of two infrared spectrum cameras for early cancer detection, Proc. SPIE. 6979, Independent Component Analyses, Wavelets, Unsupervised Nano-Biomimetic Sensors, and Neural Networks VI, 69790R.