Neural Circuits and Algorithms

2024

Neuronal temporal filters as normal mode extractors
Golkar S, Berman J, Lipshutz D, Haret R, Gollisch T, and Chklovskii D

Phys. Rev. Res. 2024

URL
The Neuron as a Direct Data-Driven Controller
Moore J, Genkin A, Tournoy M, Pughe-Sanford J, Steveninck R, and Chklovskii D

bioRxiv 2024

Abs URL

In the quest to model neuronal function amidst gaps in physiological data, a promising strategy is to develop a normative theory that interprets neuronal physiology as optimizing a computational objective. This study extends the current normative models, which primarily optimize prediction, by conceptualizing neurons as optimal feedback controllers. We posit that neurons, especially those beyond early sensory areas, act as controllers, steering their environment towards a specific desired state through their output. This environment comprises both synaptically interlinked neurons and external motor sensory feedback loops, enabling neurons to evaluate the effectiveness of their control via synaptic feedback. Utilizing the novel Direct Data-Driven Control (DD-DC) framework, we model neurons as biologically feasible controllers which implicitly identify loop dynamics, infer latent states and optimize control. Our DD-DC neuron model explains various neurophysiological phenomena: the shift from potentiation to depression in Spike-Timing-Dependent Plasticity (STDP) with its asymmetry, the duration and adaptive nature of feedforward and feedback neuronal filters, the imprecision in spike generation under constant stimulation, and the characteristic operational variability and noise in the brain. Our model presents a significant departure from the traditional, feedforward, instant-response McCulloch-Pitts-Rosenblatt neuron, offering a novel and biologically-informed fundamental unit for constructing neural networks.Competing Interest StatementThe authors have declared no competing interest.

2023

Normative and mechanistic model of an adaptive circuit for efficient encoding and feature extraction
Chapochnikov N, Pehlevan C, and Chklovskii D

Proceedings of the National Academy of Science 2023

URL
Catalyzing next-generation Artificial Intelligence through NeuroAI
Zador A, Escola S, Richards B, Ölveczky B, Bengio Y, Boahen K, Botvinick M, Chklovskii D, Churchland A, Clopath C, DiCarlo J, Ganguli S, Hawkins J, Körding K, Koulakov A, LeCun Y, Lillicrap T, Marblestone A, Olshausen B, Pouget A, Savin C, Sejnowski T, Simoncelli E, Solla S, Sussillo D, Tolias A, and Tsao D

Nature Communications 2023

URL
A linear discriminant analysis model of imbalanced associative learning in the mushroom body compartment
Lipshutz D, Kashalikar A, Farashahi S, and Chklovskii D

PLOS Computational Biology 2023

URL
Coordinated drift of receptive fields in Hebbian/anti-Hebbian network models during noisy representation learning
Qin S, Farashahi S, Lipshutz D, Sengupta A, Chklovskii D, and Pehlevan C

Nature Neuroscience 2023

URL
A normative framework for deriving neural networks with multi-compartmental neurons and non-Hebbian plasticity
Lipshutz D, Bahroun Y, Golkar S, Sengupta A, and Chklovskii D

2023

URL
Statistical whitening of neural populations with gain-modulating interneurons
Duong L, Lipshutz D, Heeger D, Chklovskii D, and Simoncelli E

2023

URL
An Out-of-Domain Synapse Detection Challenge for Microwasp Brain Connectomes
Wu J, Li Y, Gupta N, Shinomiya K, Gunn P, Polilov A, Pfister H, Chklovskii D, and Wei D

2023

URL
A complete reconstruction of the early visual system of an adult insect
Chua N, Makarova A, Gunn P, Villani S, Cohen B, Thasin M, Wu J, Shefter D, Pang S, Xu C, Hess H, Polilov A, and Chklovskii D

Current Biology 2023

Abs URL

For most model organisms in neuroscience, research into visual processing in the brain is difficult because of a lack of high-resolution maps that capture complex neuronal circuitry. The microinsect Megaphragma viggianii, because of its small size and non-trivial behavior, provides a unique opportunity for tractable whole-organism connectomics. We image its whole head using serial electron microscopy. We reconstruct its compound eye and analyze the optical properties of the ommatidia as well as the connectome of the first visual neuropil—the lamina. Compared with the fruit fly and the honeybee, Megaphragma visual system is highly simplified: it has 29 ommatidia per eye and 6 lamina neuron types. We report features that are both stereotypical among most ommatidia and specialized to some. By identifying the “barebones” circuits critical for flying insects, our results will facilitate constructing computational models of visual processing in insects.
Adaptive whitening with fast gain modulation and slow synaptic plasticity
Duong L, Simoncelli E, Chklovskii D, and Lipshutz D

Current Biology 2023

URL
Duality Principle and Biologically Plausible Learning: Connecting the Representer Theorem and Hebbian Learning
Bahroun Y, Chklovskii D, and Sengupta A

arXiv preprint arXiv:2309.16687 2023

URL
Unlocking the Potential of Similarity Matching: Scalability, Supervision and Pre-training
Bahroun Y, Sridharan S, Acharya A, Chklovskii D, and Sengupta A

arXiv preprint arXiv:2309.16687 2023

URL
An Online Algorithm for Contrastive Principal Component Analysis
Golkar S, Lipshutz D, Tesileanu T, and Chklovskii D

In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2023

URL

2022

Biologically plausible single-layer networks for nonnegative independent component analysis
Lipshutz D, Pehlevan C, and Chklovskii D

Biological Cybernetics 2022

URL
Neural Circuits for Dynamics-Based Segmentation of Time Series
Tesileanu T, Golkar S, Nasiri S, Sengupta A, and Chklovskii D

Neural Computation 2022

Abs PDF Code URL

The brain must extract behaviorally relevant latent variables from the signals streamed by the sensory organs. Such latent variables are often encoded in the dynamics that generated the signal rather than in the specific realization of the waveform. Therefore, one problem faced by the brain is to segment time series based on underlying dynamics. We present two algorithms for performing this segmentation task that are biologically plausible, which we define as acting in a streaming setting and all learning rules being local. One algorithm is model based and can be derived from an optimization problem involving a mixture of autoregressive processes. This algorithm relies on feedback in the form of a prediction error and can also be used for forecasting future samples. In some brain regions, such as the retina, the feedback connections necessary to use the prediction error for learning are absent. For this case, we propose a second, model-free algorithm that uses a running estimate of the autocorrelation structure of the signal to perform the segmentation. We show that both algorithms do well when tasked with segmenting signals drawn from autoregressive models with piecewise-constant parameters. In particular, the segmentation accuracy is similar to that obtained from oracle-like methods in which the ground-truth parameters of the autoregressive models are known. We also test our methods on data sets generated by alternating snippets of voice recordings. We provide implementations of our algorithms at https://github.com/ttesileanu/bio-time-series.
Neural Circuits for Dynamics-Based Segmentation of Time Series
Teşileanu T, Golkar S, Nasiri S, Sengupta A, and Chklovskii D

Neural Computation 2022

Abs URL

The brain must extract behaviorally relevant latent variables from the signals streamed by the sensory organs. Such latent variables are often encoded in the dynamics that generated the signal rather than in the specific realization of the waveform. Therefore, one problem faced by the brain is to segment time series based on underlying dynamics. We present two algorithms for performing this segmentation task that are biologically plausible, which we define as acting in a streaming setting and all learning rules being local. One algorithm is model based and can be derived from an optimization problem involving a mixture of autoregressive processes. This algorithm relies on feedback in the form of a prediction error and can also be used for forecasting future samples. In some brain regions, such as the retina, the feedback connections necessary to use the prediction error for learning are absent. For this case, we propose a second, model-free algorithm that uses a running estimate of the autocorrelation structure of the signal to perform the segmentation. We show that both algorithms do well when tasked with segmenting signals drawn from autoregressive models with piecewise-constant parameters. In particular, the segmentation accuracy is similar to that obtained from oracle-like methods in which the ground-truth parameters of the autoregressive models are known. We also test our methods on data sets generated by alternating snippets of voice recordings. We provide implementations of our algorithms at https://github.com/ttesileanu/bio-time-series.
Biological Learning of Irreducible Representations of Commuting Transformations
Genkin A, Lipshutz D, Golkar S, Tesileanu T, and Chklovskii D

In Advances in Neural Information Processing Systems 2022

URL
Constrained Predictive Coding as a Biologically Plausible Model of the Cortical Hierarchy
Golkar S, Tesileanu T, Bahroun Y, Sengupta A, and Chklovskii D

In Advances in Neural Information Processing Systems 2022

URL
An online algorithm for contrastive Principal Component Analysis
Golkar S, Lipshutz D, Tesileanu T, and Chklovskii D

2022

URL
Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptation
Lipshutz D, Pehlevan C, and Chklovskii D

2022

URL

2021

A Biologically Plausible Neural Network for Multichannel Canonical Correlation Analysis
Lipshutz D, Bahroun Y, Golkar S, Sengupta A, and Chklovskii D

Neural Computation 2021

Abs PDF URL

Cortical pyramidal neurons receive inputs from multiple distinct neural populations and integrate these inputs in separate dendritic compartments. We explore the possibility that cortical microcircuits implement canonical correlation analysis (CCA), an unsupervised learning method that projects the inputs onto a common subspace so as to maximize the correlations between the projections. To this end, we seek a multichannel CCA algorithm that can be implemented in a biologically plausible neural network. For biological plausibility, we require that the network operates in the online setting and its synaptic update rules are local. Starting from a novel CCA objective function, we derive an online optimization algorithm whose optimization steps can be implemented in a single-layer neural network with multicompartmental neurons and local non-Hebbian learning rules. We also derive an extension of our online CCA algorithm with adaptive output rank and output whitening. Interestingly, the extension maps onto a neural network whose neural architecture and synaptic updates resemble neural circuitry and non-Hebbian plasticity observed in the cortex.
Neural optimal feedback control with local learning rules
Friedrich J, Golkar S, Farashahi S, Genkin A, Sengupta A, and Chklovskii D

In Advances in Neural Information Processing Systems 2021

Abs PDF Code URL

A major problem in motor control is understanding how the brain plans and executes proper movements in the face of delayed and noisy stimuli. A prominent framework for addressing such control problems is Optimal Feedback Control (OFC). OFC generates control actions that optimize behaviorally relevant criteria by integrating noisy sensory stimuli and the predictions of an internal model using the Kalman filter or its extensions. However, a satisfactory neural model of Kalman filtering and control is lacking because existing proposals have the following limitations: not considering the delay of sensory feedback, training in alternating phases, requiring knowledge of the noise covariance matrices, as well as that of systems dynamics. Moreover, the majority of these studies considered Kalman filtering in isolation, and not jointly with control. To address these shortcomings, we introduce a novel online algorithm which combines adaptive Kalman filtering with a model free control approach (i.e., policy gradient algorithm). We implement this algorithm in a biologically plausible neural network with local synaptic plasticity rules. This network, with local synaptic plasticity rules, performs system identification, Kalman filtering and control with delayed noisy sensory feedback. This network performs system identification and Kalman filtering, without the need for multiple phases with distinct update rules or the knowledge of the noise covariances. It can perform state estimation with delayed sensory feedback, with the help of an internal model. It learns the control policy without requiring any knowledge of the dynamics, thus avoiding the need for weight transport. In this way, our implementation of OFC solves the credit assignment problem needed to produce the appropriate sensory-motor control in the presence of stimulus delay.
VolPy: Automated and scalable analysis pipelines for voltage imaging datasets
Cai C, Friedrich J, Singh A, Eybposh M, Pnevmatikakis E, Podgorski K, and Giovannucci A

PLoS Computational Biology 2021

Abs PDF URL

Voltage imaging enables monitoring neural activity at sub-millisecond and sub-cellular scale, unlocking the study of subthreshold activity, synchrony, and network dynamics with unprecedented spatio-temporal resolution. However, high data rates (>800MB/s) and low signal-to-noise ratios create bottlenecks for analyzing such datasets. Here we present VolPy, an automated and scalable pipeline to pre-process voltage imaging datasets. VolPy features motion correction, memory mapping, automated segmentation, denoising and spike extraction, all built on a highly parallelizable, modular, and extensible framework optimized for memory and speed. To aid automated segmentation, we introduce a corpus of 24 manually annotated datasets from different preparations, brain areas and voltage indicators. We benchmark VolPy against ground truth segmentation, simulations and electrophysiology recordings, and we compare its performance with existing algorithms in detecting spikes. Our results indicate that VolPy’s performance in spike extraction and scalability are state-of-the-art.
Online analysis of microendoscopic 1-photon calcium imaging data streams
Friedrich J, Giovannucci A, and Pnevmatikakis E

PLoS Computational Biology 2021

Abs PDF URL

In vivo calcium imaging through microendoscopic lenses enables imaging of neuronal populations deep within the brains of freely moving animals. Previously, a constrained matrix factorization approach (CNMF-E) has been suggested to extract single-neuronal activity from microendoscopic data. However, this approach relies on offline batch processing of the entire video data and is demanding both in terms of computing and memory requirements. These drawbacks prevent its applicability to the analysis of large datasets and closed-loop experimental settings. Here we address both issues by introducing two different online algorithms for extracting neuronal activity from streaming microendoscopic data. Our first algorithm, \on, presents an online adaptation of the CNMF-E algorithm, which dramatically reduces its memory and computation requirements. Our second algorithm proposes a convolution-based background model for microendoscopic data that enables even faster (real time) processing. Our approach is modular and can be combined with existing online motion artifact correction and activity deconvolution methods to provide a highly scalable pipeline for microendoscopic data analysis. We apply our algorithms on four previously published typical experimental datasets and show that they yield similar high-quality results as the popular offline approach, but outperform it with regard to computing time and memory requirements. They can be used instead of CNMF-E to process pre-recorded data with boosted speeds and dramatically reduced memory requirements. Further, they newly enable online analysis of live-streaming data even on a laptop.
Bridging the Gap: Point Clouds for Merging Neurons in Connectomics
Berman J, Chklovskii D, and Wu J

CoRR 2021

Abs PDF URL

In the field of Connectomics, a primary problem is that of 3D neuron segmentation. Although deep learning-based methods have achieved remarkable accuracy, errors still exist, especially in regions with image defects. One common type of defect is that of consecutive missing image sections. Here, data is lost along some axis, and the resulting neuron segmentations are split across the gap. To address this problem, we propose a novel method based on point cloud representations of neurons. We formulate the problem as a classification problem and train CurveNet, a state-of-the-art point cloud classification model, to identify which neurons should be merged. We show that our method not only performs strongly but also scales reasonably to gaps well beyond what other methods have attempted to address. Additionally, our point cloud representations are highly efficient in terms of data, maintaining high performance with an amount of data that would be unfeasible for other methods. We believe that this is an indicator of the viability of using point cloud representations for other proofreading tasks.
Neural circuits for dynamics-based segmentation of time series
Tesileanu T, Golkar S, Nasiri S, Sengupta A, and Chklovskii D

Neural Computation 2021

Abs Bib PDF URL

The brain must extract behaviorally relevant latent variables from the signals streamed by the sensory organs. Such latent variables are often encoded in the dynamics that generated the signal rather than in the specific realization of the waveform. Therefore, one problem faced by the brain is to segment time series based on underlying dynamics. We present two algorithms for performing this segmentation task that are biologically plausible, which we define as acting in a streaming setting and all learning rules being local. One algorithm is model based and can be derived from an optimization problem involving a mixture of autoregressive processes. This algorithm relies on feedback in the form of a prediction error and can also be used for forecasting future samples. In some brain regions, such as the retina, the feedback connections necessary to use the prediction error for learning are absent. For this case, we propose a second, model-free algorithm that uses a running estimate of the autocorrelation structure of the signal to perform the segmentation. We show that both algorithms do well when tasked with segmenting signals drawn from autoregressive models with piecewise-constant parameters. In particular, the segmentation accuracy is similar to that obtained from oracle-like methods in which the ground-truth parameters of the autoregressive models are known. We also test our methods on data sets generated by alternating snippets of voice recordings. We provide implementations of our algorithms at https://github.com/ttesileanu/bio-time-series.
@article{Tesileanu2021, img = {tibi_seq_seg.png}, bibtex_show = {true}, selected = {true}, archiveprefix = {arXiv}, arxivid = {2104.11852}, author = {Tesileanu, Tiberiu and Golkar, Siavash and Nasiri, Samaneh and Sengupta, Anirvan M. and Chklovskii, Dmitri B.}, doi = {10.1162/neco_a_01476}, journal = {Neural Computation}, eprint = {2104.11852}, pages = {1--34}, title = {{Neural circuits for dynamics-based segmentation of time series}}, related = {Theory}, url = {http://arxiv.org/abs/2104.11852}, pdf = {https://arxiv.org/pdf/2104.11852.pdf}, year = {2021} }
A Normative and Biologically Plausible Algorithm for Independent Component Analysis
Bahroun Y, Chklovskii D, and Sengupta A

In Advances in Neural Information Processing Systems 2021

URL
Normative and mechanistic model of an adaptive circuit for efficient encoding and feature extraction
Chapochnikov N, Pehlevan C, and Chklovskii D

bioRxiv 2021

Abs URL

One major question in neuroscience is how to relate connectomes to neural activity, circuit function, and learning. We offer an answer in the peripheral olfactory circuit of the Drosophila larva, composed of olfactory receptor neurons (ORNs) connected through feedback loops with interconnected inhibitory local neurons (LNs). We combine structural and activity data and, using a holistic normative framework based on similarity-matching, we propose a biologically plausible mechanistic model of the circuit. Our model predicts the ORN \textrightarrow LN synaptic weights found in the connectome and demonstrate that they reflect correlations in ORN activity patterns. Additionally, our model explains the relation between ORN \textrightarrow LN and LN – LN synaptic weight and the arising of different LN types. This global synaptic organization can autonomously arise through Hebbian plasticity, and thus allows the circuit to adapt to different environments in an unsupervised manner. Functionally, we propose LNs extract redundant input correlations and dampen them in ORNs, thus partially whitening and normalizing the stimulus representations in ORNs. Our work proposes a comprehensive framework to combine structure, activity, function, and learning, and uncovers a general and potent circuit motif that can learn and extract significant input features and render stimulus representations more efficient.Significance The brain represents information with patterns of neural activity. At the periphery, due to the properties of the external world and of encoding neurons, these patterns contain correlations, which are detrimental for stimulus discrimination. We study the peripheral olfactory neural circuit of the Drosophila larva, that preprocesses neural representations before relaying them to higher brain areas. A comprehensive understanding of this preprocessing is, however, lacking. Here, we propose a mechanistic and normative framework describing the function of the circuit and predict the circuit’s synaptic organization based on the circuit’s input neural activity. We show how the circuit can autonomously adapt to different environments, extracts stimulus features, and decorrelate and normalize input representations, which facilitates odor discrimination downstream.Competing Interest StatementThe authors have declared no competing interest.
Small brains for big science
Makarova A, Polilov A, and Chklovskii D

Current Opinion in Neurobiology 2021

Abs URL

As the study of the human brain is complicated by its sheer scale, complexity, and impracticality of invasive experiments, neuroscience research has long relied on model organisms. The brains of macaque, mouse, zebrafish, fruit fly, nematode, and others have yielded many secrets that advanced our understanding of the human brain. Here, we propose that adding miniature insects to this collection would reduce the costs and accelerate brain research. The smallest insects occupy a special place among miniature animals: despite their body sizes, comparable to unicellular organisms, they retain complex brains that include thousands of neurons. Their brains possess the advantages of those in insects, such as neuronal identifiability and the connectome stereotypy, yet are smaller and hence easier to map and understand. Finally, the brains of miniature insects offer insights into the evolution of brain design.

2020

Neuronal Gaussian Process Regression
Friedrich J

In Advances in Neural Information Processing Systems 2020

Abs PDF Code URL

The brain takes uncertainty intrinsic to our world into account. For example, associating spatial locations with rewards requires to predict not only expected reward at new spatial locations but also its uncertainty to avoid catastrophic events and forage safely. A powerful and flexible framework for nonlinear regression that takes uncertainty into account in a principled Bayesian manner is Gaussian process (GP) regression. Here I propose that the brain implements GP regression and present neural networks (NNs) for it. First layer neurons, e.g. hippocampal place cells, have tuning curves that correspond to evaluations of the GP kernel. Output neurons explicitly and distinctively encode predictive mean and variance, as observed in orbitofrontal cortex (OFC) for the case of reward prediction. Because the weights of a NN implementing exact GP regression do not arise with biological plasticity rules, I present approximations to obtain local (anti-)Hebbian synaptic learning rules. The resulting neuronal network approximates the full GP well compared to popular sparse GP approximations and achieves comparable predictive performance.
A simple normative network approximates local non-Hebbian learning in the cortex
Golkar S, Lipshutz D, Bahroun Y, Sengupta A, and Chklovskii D

Advances in Neural Information Processing Systems 2020

Abs URL

To guide behavior, the brain extracts relevant features from high-dimensional data streamed by sensory organs. Neuroscience experiments demonstrate that the processing of sensory inputs by cortical neurons is modulated by instructive signals which provide context and task-relevant information. Here, adopting a normative approach, we model these instructive signals as supervisory inputs guiding the projection of the feedforward data. Mathematically, we start with a family of Reduced-Rank Regression (RRR) objective functions which include Reduced Rank (minimum) Mean Square Error (RRMSE) and Canonical Correlation Analysis (CCA), and derive novel offline and online optimization algorithms, which we call Bio-RRR. The online algorithms can be implemented by neural networks whose synaptic learning rules resemble calcium plateau potential dependent plasticity observed in the cortex. We detail how, in our model, the calcium plateau potential can be interpreted as a backpropagating error signal. We demonstrate that, despite relying exclusively on biologically plausible local learning rules, our algorithms perform competitively with existing implementations of RRMSE and CCA.
A biologically plausible neural network for slow feature analysis
Lipshutz D, Windolf C, Golkar S, and Chklovskii D

Advances in Neural Information Processing Systems 2020

Abs URL

Learning latent features from time series data is an important problem in both machine learning and brain function. One approach, called Slow Feature Analysis (SFA), leverages the slowness of many salient features relative to the rapidly varying input signals. Furthermore, when trained on naturalistic stimuli, SFA reproduces interesting properties of cells in the primary visual cortex and hippocampus, suggesting that the brain uses temporal slowness as a computational principle for learning latent features. However, despite the potential relevance of SFA for modeling brain function, there is currently no SFA algorithm with a biologically plausible neural network implementation, by which we mean an algorithm operates in the online setting and can be mapped onto a neural network with local synaptic updates. In this work, starting from an SFA objective, we derive an SFA algorithm, called Bio-SFA, with a biologically plausible neural network implementation. We validate Bio-SFA on naturalistic stimuli.

2019

CaImAn: An open source tool for scalable Calcium Imaging data Analysis
Giovannucci A, Friedrich J, Gunn P, Kalfon J, Brown B, Koay S, Taxidis J, Najafi F, Gauthier J, Zhou P, Khakh B, Tank D, Chklovskii D, and Pnevmatikakis E

eLife 2019

Abs Code URL

Advances in fluorescence microscopy enable monitoring larger brain areas in-vivo with finer time resolution. The resulting data rates require reproducible analysis pipelines that are reliable, fully automated, and scalable to datasets generated over the course of months. We present CaImAn, an open-source library for calcium imaging data analysis. CaImAn provides automatic and scalable methods to address problems common to pre-processing, including motion correction, neural activity identification, and registration across different sessions of data collection. It does this while requiring minimal user intervention, with good scalability on computers ranging from laptops to high-performance computing clusters. CaImAn is suitable for two-photon and one-photon imaging, and also enables real-time analysis on streaming data. To benchmark the performance of CaImAn we collected and combined a corpus of manual annotations from multiple labelers on nine mouse two-photon datasets. We demonstrate that CaImAn achieves near-human performance in detecting locations of active neurons.
Neuroscience-Inspired Online Unsupervised Learning Algorithms: Artificial Neural Networks
Pehlevan C, and Chklovskii D

IEEE Signal Processing Magazine 2019

Abs PDF URL

Inventors of the original artificial neural networks (ANNs) derived their inspiration from biology [1]. However, today, most ANNs, such as backpropagation-based convolutional deeplearning networks, resemble natural NNs only superficially. Given that, on some tasks, such ANNs achieve human or even superhuman performance, why should one care about such dissimilarity with natural NNs? The algorithms of natural NNs are relevant if one’s goal is not just to outperform humans on certain tasks but to develop general-purpose artificial intelligence rivaling that of a human. As contemporary ANNs are far from achieving this goal and natural NNs, by definition, achieve it, natural NNs must contain some ’secret sauce’ that ANNs lack. This is why we need to understand the algorithms implemented by natural NNs.

2018

Why Do Similarity Matching Objectives Lead to Hebbian/Anti-Hebbian Networks?
Pehlevan C, Sengupta A, and Chklovskii D

Neural Computation 2018

PDF URL
Manifold-tiling Localized Receptive Fields are Optimal in Similarity-preserving Neural Networks
Sengupta A, Tepper M, Pehlevan C, Genkin A, and Chklovskii D

bioRxiv 2018

Abs PDF URL

Many neurons in the brain, such as place cells in the rodent hippocampus, have localized receptive fields, i.e., they respond to a small neighborhood of stimulus space. What is the functional significance of such representations and how can they arise? Here, we propose that localized receptive fields emerge in similarity-preserving networks of rectifying neurons that learn low-dimensional manifolds populated by sensory inputs. Numerical simulations of such networks on standard datasets yield manifold-tiling localized receptive fields. More generally, we show analytically that, for data lying on symmetric manifolds, optimal solutions of objectives, from which similarity-preserving networks are derived, have localized receptive fields. Therefore, nonnegative similarity-preserving mapping (NSM) implemented by neural networks can model representations of continuous manifolds in the brain.
A clustering neural network model of insect olfaction
Pehlevan C, Genkin A, and Chklovskii D

bioRxiv 2018

Abs PDF URL

A key step in insect olfaction is the transformation of a dense representation of odors in a small population of neurons - projection neurons (PNs) of the antennal lobe - into a sparse representation in a much larger population of neurons -Kenyon cells (KCs) of the mushroom body. What computational purpose does this transformation serve? We propose that the PN-KC network implements an online clustering algorithm which we derive from the k-means cost function. The vector of PN-KC synaptic weights converging onto a given KC represents the corresponding cluster centroid. KC activities represent attribution indices, i.e. the degree to which a given odor presentation is attributed to each cluster. Remarkably, such clustering view of the PN-KC circuit naturally accounts for several of its salient features. First, attribution indices are nonnegative thus rationalizing rectification in KCs. Second, the constraint on the total sum of attribution indices for each presentation is enforced by a Lagrange multiplier identified with the activity of a single inhibitory interneuron reciprocally connected with KCs. Third, the soft-clustering version of our algorithm reproduces observed sparsity and overcompleteness of the KC representation which may optimize supervised classification downstream.