Nicholas C. Makris

Highly detailed design knowledge of a Golden-Period Stradivarius violin, the CT-scanned Titian, is incorporated into a computational violin with tensioned strings that are plucked. It includes full compressible aerodynamic interaction between violin structure and air. The resulting computational instrument plays music, as demonstrated with measures of a Bach solo violin fugue and the song Daisy Bell, as received by an audience member at any range and direction from the instrument. Violin design is then evaluated directly from the computational instrument’s plucked string output in an efficient and controlled computational manner, potentially saving significant time and resources in lutherie and making possible new forms of musical expression. To explore quantitative violin performance metrics, power efficiency across plucked, open and fingered notes is computed and found to exhibit significant departures from uniformity. Commensurate playing compensation is shown to be necessary to achieve uniform power output across notes. Via high resolution spatial computations on and around the violin, significant variations are found in acoustic power flow from key violin components across frequency. These are found to propagate outward and lead to variations in radiated sound intensity at the position of a listening player or audience member over frequency, which can be large for small changes in either.

The ability of natural environmental sound to stimulate seeds and seedlings sufficiently to foster growth has not been previously demonstrated or quantified. To study this, rain sound is a logical starting point. Rain produces extremely high amplitude sound pressure with commensurate particle displacements in the upper soil, puddles and wetlands where many plant seeds germinate. Experiments were conducted with controlled rain drops impacting soil and shallow water puddles containing submerged seeds of rice (oryza sativa). Germination rates were measured as the peak sound pressure of drop impact was varied. The displacements of micro-meter-scale statoliths relative to the structure of specialized seed cells that sense gravitational direction were estimated as a function of the controlled rain sound forcing. The results here indicate rice and related seed types can sense the sound of rain impacting the soil or water surface above them and respond by accelerating germination at depths where impulsive rain sound is sufficiently intense to intermittently shake statoliths from contact with cell membrane receptors and trigger gravitropic growth mechanisms. The ability to perceive rain sound and respond with accelerated germination is found to be roughly limited to the relatively shallow depths that are also beneficial to seedling survival.

Accurate, real-time underwater acoustic sensing is essential for marine biological and ecological studies, compliance monitoring, maritime surveillance, and defense applications. In such applications, densely-populated coherent hydrophone arrays are emerging as important tools for ocean acoustic sensing since they are capable of (1) signal-to-noise ratio (SNR) enhancement, (2) directional sensing and bearing estimation via beamforming, and (3) source spatial localization over instantaneous wide areas. However, capturing high-resolution multichannel acoustic data from deep sea environments presents challenges in power management, bandwidth, pressure tolerance, and data integrity. This paper addresses these challenges by investigating designs and approaches for a compact, low-power, and scalable electronic system capable of reliably digitizing and streaming acoustic data from multiple elements of a coherent hydrophone array over Ethernet in real-time, without relying on bandwidth-limited serial protocols. At the core of the system is a high-performance 24-bit multichannel Analog-to-Digital Converter (ADC) interfaced with a Field-Programmable Gate Array (FPGA). In the future, the system will be designed to be deployable in both air and underwater multi-element acoustic array sensor systems, including coherent hydrophone arrays requiring compact small form factor multichannel ADCs for embedding in oil-filled tubing. Here we demonstrate and validate the design approach by acoustic testing in-air, including waveform inspection and Ethernet frame verification. This platform enables continuous, high-throughput, and low-latency streaming of synchronized multichannel acoustic data from underwater arrays. By eliminating external control and minimizing power consumption, it paves the way for scalable plug-and-play ocean sensing systems designed for deployment in extreme environments.

Humpback whale breathing-related sounds were recorded on elements of a coherent hydrophone array subaperture deployed vertically at the Great South Channel on the US Northeastern continental shelf in Fall 2021, where half of the hydrophones were in-air and the rest submerged underwater. In-air hydrophones recorded breathing sounds with approximately 2.5 s duration, but smaller bandwidths compared to underwater hydrophones where signal energies extended beyond 50 kHz, and a mean underwater source level of 161 ± 4 dB re 1 μPa at 1 m, based on measurements at 22.9 m. The underwater recorded humpback whale breathing sound spectra displayed a broadband dip centered at 15.7 kHz, with approximately 400 Hz half-power bandwidth, likely caused by attenuation from propagation through pulsating air bubbles. The air bubble radius for natural frequency of oscillations at 15.7 kHz is estimated to be 0.205-0.21 mm. These bubbles are capable of removing energy from the forward propagated humpback breathing sounds via resonance absorption most pronounced at and near bubble natural oscillation frequency. Humpback whale distances from the vertically deployed hydrophones are estimated and tracked by matching the curved nonlinear travel-time wavefront of its breathing sounds, since the whale was in the near-field of the subarray.

A large-aperture densely populated coherent hydrophone array system was towed at multiple locations off Norway encompassing regions above and below the Artic Circle during an experiment in spring 2014. Vocalization signals up to 4 kHz from diverse marine mammals species were received over multiple diel cycles at each measurement site off Alesund, Loften, and the Northern Finnmark region of the Norwegian and Barents Seas. Here we compare and contrast the species-dependent marine mammal vocalization temporal and spatial distributions for the three regions. Data from the coherent hydrophone array system are automatically processed to enable signal detections, bearing estimation via beamforming, and time-frequency feature extraction via pitch tracking. Bearing-time trajectories of signal detections after automatic classfication are first extensively verified by visual inspection of select spectrograms. Verified marine mammal vocalization bearing-time trajectories are next localized via the moving array triangulation technique and mapped onto geographic space, including error bounds for range and cross-range estimates. Vocalizations belonging to baleen whale species include fin, humpback and minke, and tooth whale species include sperm and beluga. The species-dependent diel, diurnal and nocturnal call rates, as well as call spatial distributions provide insights into their behavior and interactions in the undersea environment.

Humpback whale breathing-related sounds were recorded on elements of a coherent hydrophone array subaperture deployed vertically at the Great South Channel on the US Northeastern continental shelf in Fall 2021, where half of the hydrophones were in-air and the rest submerged underwater. In-air hydrophones recorded breathing sounds with approximately 2.5 s duration, but smaller bandwidths compared to underwater hydrophones where signal energies extended beyond 50 kHz, and a mean underwater source level of 161 ± 4 dB re 1 μPa at 1 m, based on measurements at 22.9 m. The underwater recorded humpback whale breathing sound spectra displayed a broadband dip centered at 15.7 kHz, with approximately 400 Hz half-power bandwidth, likely caused by attenuation from propagation through pulsating air bubbles. The air bubble radius for natural frequency of oscillations at 15.7 kHz is estimated to be 0.205–0.21 mm. These bubbles are capable of removing energy from the forward propagated humpback breathing sounds via resonance absorption most pronounced at and near bubble natural oscillation frequency. Humpback whale distances from the vertically deployed hydrophones are estimated and tracked by matching the curved nonlinear travel-time wavefront of its breathing sounds, since the whale was in the near-field of the subarray.

Sensing limitations have impeded knowledge about how individual predator-prey interactions build to organized multi-species group behaviour across an ecosystem. Population densities of overlapping interacting oceanic fish predator and prey species, however, can be instantaneously distinguished and quantified from roughly the elemental individual to spatial scales spanning thousands of square kilometres by wide-area multispectral underwater-acoustic sensing, as shown here. This enables fundamental mechanisms behind large-scale ordered predator-prey interactions to be investigated. Critical population densities that transition random individual behaviour to ordered group behaviour are found to rapidly propagate to form vast adversarial prey and predator shoals of capelin and surrounding cod in the Barents Sea Arctic ecosystem for these keystone species. This leads to a sudden major shift in predator-prey balance. Only a small change in local behaviour triggers the shift due to an unstable equilibrium. Such unstable equilibria and associated balance shifts at predation hotspots are often overlooked as blind spots in present ocean ecosystem monitoring and assessment due to use of highly undersampled spatio-temporal sampling methods.

Developing automatic algorithms for real-time monitoring of underwater acoustic events is essential in ocean acoustic applications. Most previous ocean acoustic ecosystem monitoring studies are non-real-time, focusing on data received on a single hydrophone or a specific analysis, such as bearing estimation or detection, without considering the full end-to-end analysis system. Here, we develop a unified framework for real-time ocean acoustic data analysis including beamforming, detection, bearing estimation, and classification of transient underwater acoustic events. To detect sound sources, thresholding on computed mel-scale per-channel energy normalization (PCEN) is applied, followed by morphological image opening to extract pixels with significant intensities. Next, connected component analysis is applied for grouping pixel detections. The bearing of signal detections is next estimated via nonmaximum suppression (NMS) of 3-D stacked beamformed spectrogram imageries. To classify a variety of whale species from their calls, time-frequency features are extracted from each detected signal’s beamformed power spectrogram. These features are next applied to train three classifiers, including support vector machine (SVM), neural networks, and random forest (RF), to classify six whale vocalization categories: Fin, Sei, Unidentified Baleen, Minke, Humpback, and general Odontocetes. Best results are obtained with the RF classifier, which achieved 96.7% accuracy and 87.5% F1 score. A variety of accelerating approaches and fast algorithms are implemented to run on GPU. During an experiment in the U.S. Northeast coast in September 2021, the software and hardware advances developed here were used for near real-time analysis of underwater acoustic data received by Northeastern University’s in-house fabricated 160-element coherent hydrophone array system.

Directional soundscaping is an efficient approach for examining marine ecosystems since it allows the study of living organisms, their behavior, and temporo-spatial interactions with other natural and man-made objects underwater, useful for ecosystem monitoring during offshore energy activities, maritime surveillance, and defense. A large aperture coherent hydrophone array was employed for remote sensing in the Norwegian and Barents Seas in Spring 2014. Extensive analysis have been conducted in post-processing of recorded hydrophone array data for automatic detection, bearing estimation, and classification of signals by different sound producers, such as whale calls, ship radiated noise, and fish sounds. Directional soundscaping through passive ocean acoustic waveguide remote sensing (POAWRS) provides bearing-time trajectories of signal detections that can be applied for geographical mapping. The received marine mammal sounds include vocalizations from baleen whales such as humpback, fin, and minke, and toothed whales, such as beluga, pilot, and sperm. We provide an insight to the vocalizations and behaviorial patterns of these whales in the Lofoten and Northern Finmark regions. The relative contributions of distinct sound sources, including whales, fish, and ships, to the directional soundscapes are quantified. We also examine their temporo-spatial dependences via geographic mapping of acoustic signals from distinct sound producers.

The vocalization behavior of humpback whales in the Norwegian and Barents Seas is examined based on recordings of a large-aperture, densely-populated coherent hydrophone array system. The passive ocean acoustic waveguide remote sensing (POAWRS) technique is employed to provide detection, bearing-time estimation, time-frequency characterization and classification of the humpback whale vocalizations. The song vocalizations, composed of highly structured and repeatable set of phrases, were detected throughout the diel cycle between February 18 to March 8, 2014. The beamformed spectrograms of the detected humpback vocalizations are classified as song sequences based on inter-pulse intervals and time-frequency characteristics, verified by visual inspection. The song structure is compared for humpback whale vocalizations recorded at three distinct regions off the Norwegian coast, Alesund, Lofoten and Northern Finmark. Multiple bearing-time trajectories for humpback songs were simultaneously observed indicating multiple singers present at each measurement site. Humpback whale received call rates and temporo-spatial distributions are compared across the three measurement sites. Geographic mapping of humpback whale calls from their bearing-time trajectories is accomplished via the moving array triangulation technique.

Leveraging data acquired using a 160-element coherent hydrophone array deployed in the Norwegian and Barents Seas during spring 2014, and the passive ocean acoustic waveguide remote sensing (POAWRS) technique is employed to enable instantaneous wide-area monitoring of marine mammal vocalizations over expanses exceeding 100 km in diameter. The vocalization behavior of diverse marine mammal species including Fin, Humpback, Minke, Sperm, and Beluga whales are analyzed, quantifying time-frequency characteristics and call patterns from their vocalization signals present in high-resolution beamformed power spectrograms. Previously developed automatic detection and machine learning algorithms are employed for clustering and classification of underwater acoustic events, facilitating the subsequent manual audio-visual inspection of whale calls. Bearing-time trajectories of repetitive species-specific vocalizations signals are utilized to estimate locations of the whale calls and hence their temporo-spatial distributions. Through comparative studies, we assess the presence and abundance of marine mammal species-specific vocalization signals in three distinct coastal regions off Norway, namely, Alesund, Lofoten, and Northern Finmark. In addition, we investigate correspondences of marine mammal distributions with concurrently imaged instantaneous wide-area fish distributions from distinct fish species to elucidate potential predator-prey interactions and habitat preferences.

Underwater acoustic monitoring with a large-aperture coherent hydrophone array is advantageous because it enhances signal-to-noise ratio of received signals, provides estimates of signal bearing, and enhances signal detection ranges by one to two orders of magnitude over that of a single hydrophone. A large aperture coherent hydrophone array system comprising >160 elements has been developed inhouse at Northeastern University. The overall acoustic aperture length is 192 m with array elements nested into multiple uniformly spaced or log spaced subapertures. Hydrophones with integrated broadband pre-amplifiers designed with a linear frequency response from 10 Hz to 50 kHz send differential pair amplified and filtered analog signals to multiple 24-bit, 32-channel analog-to-digital converters with sampling rate that is programmable up to 100 kHz per channel. Array internals are designed using field replaceable pressure tolerant components verified by pressure chamber testing. Forward and aft modules are equipped with non-acoustic sensor elements to provide depth, heading, pitch, roll and temperature measurements. Acoustic aperture telemetry is user datagram protocol (UDP) converted to single-mode fiber for transmission along 600 m of faired tow cable to a shipboard data acquisition system. Examples of passive acoustic data from array deployment in the US Northeast coast are presented illustrating array capabilities.

Efficient resolution of natural light and sound intensity is essential for organisms, systems and machines that rely on visual and auditory sensory perception to survive or function effectively in their environment. This resolution obeys Weber’s Law when the smallest resolvable change, a just-noticeable-difference, grows in direct proportion to the stimulus. Here, Weber’s Law is found to be a consequence of attaining the theoretical minimum mean-square error possible, the Cramer–Rao lower bound, in resolving the intensity of naturally scintillating light and sound. The finding is based on statistics from thousands of measurements of naturally scintillating environmental light and sound signals. Remarkably, just-noticeable-differences in light and sound intensity measured over decades of psychophysical experiments with artificial sources are also found to approximately attain the respective Cramer–Rao lower bounds. Human intensity resolution is in this way optimally adapted to the natural scintillation of light and sound. Pattern recognition by simple matched-filter correlation between measured and hypothetical images cancels natural scintillation. For intensity perception obeying Weber’s Law, this is found to be advantageous and statistically optimal because perceived scintillation is independent of the underlying signal pattern. A small visual patch change or acoustic signature truncation is shown to be lost in natural signal-dependent fluctuations if perception with constant intensity resolution is attempted.

Ocean Acoustic Waveguide Remote Sensing (OAWRS) enables fish population density distributions to be instantaneously quantified and continuously monitored over wide areas. Returns from seafloor geology can also be received as background or clutter by OAWRS when insufficient fish populations are present in any region. Given the large spatial regions that fish inhabit and roam over, it is important to develop automatic methods for determining whether fish are present at any pixel in an OAWRS image so that their population distributions, migrations and behaviour can be efficiently analyzed and monitored in large data sets. Here, a statistically optimal automated approach for distinguishing fish from seafloor geology in OAWRS imagery is demonstrated with Neyman–Pearson hypothesis testing which provides the highest true-positive classification rate for a given false-positive rate. Multispectral OAWRS images of large herring shoals during spawning migration to Georges Bank are analyzed. Automated Neyman-Pearson hypothesis testing is shown to accurately distinguish fish from seafloor geology through their differing spectral responses at any space and time pixel in OAWRS imagery. These spectral differences are most dramatic in the vicinity of swimbladder resonances of the fish probed by OAWRS. When such significantly different spectral dependencies exist between fish and geologic scattering, the approach presented provides an instantaneous, reliable and statistically optimal means of automatically distinguishing fish from seafloor geology at any spatial pixel in wide-area OAWRS images. Employing Kullback–Leibler divergence or the relative entropy in bits from Information Theory is shown to also enable automatic discrimination of fish from seafloor by their distinct statistical scattering properties across sensing frequency, but without the statistical optimal properties of the Neyman–Pearson approach.