Concepts of Neural Networks (CSA203)

📘 Syllabus (CSA203 - Theory)

• Unit 1: Introduction
  • A. Introduction, Motivation and History, Components of a Neuron-synapses, dendrite, cell nucleus, axon.
  • B. Important Terminologies of ANNs: Propagation function, Activation function, output function, Components of Artificial Neural Network: common activation functions, network topologies- feed forward, recurrent networks, completely linked networks.
  • C. Neuron Activation order: Synchronous activation, asynchronous activation, Communication with the outside world: input and output of data in and from neural networks
• Unit 2: Learning Paradigms
  • A. Learning Paradigms and their real Applications, Unsupervised learning and Supervised learning, Reinforcement learning, Offline and online learning and their applications based on real life problems.
  • B. Training patterns and teaching inputs, use of training samples, data set split into training, validation and testing data, Implication of splitting of data set, Learning curves and their importance in diagnostics.
  • C. Gradient optimization procedures, Hebbian learning rule.
• Unit 3: The Perceptron, Backpropagation and its variants
  • A. Single layer Perceptron network, Perceptron Learning Algorithm and convergence theorem, Delta rule as a gradient based learning strategy, Limitations of Single Layer Perceptron network.
  • B. Multilayer Perceptron Network, Backpropagation learning and its applications.
  • C. Analysing effect of learning rate on learning process, Variants of Back propagation algorithm.
• Unit 4: Radial Basis Function Neural Networks
  • A. Components & Structure of an RBF network, Information processing of an RBF network, Information Processing in RBF neurons, analytical thoughts prior to training.
  • B. Equation system and gradient strategies for training, Growing RBF Networks, comparison of RBF Networks and Multilayer Perceptrons.
  • C. Recurrent Neural Networks: Jordan networks, Elman Networks, Training Recurrent neural networks.
• Unit 5: Unsupervised Learning Network Paradigms
  • A. Self-organizing feature maps, structure of a self-organizing feature map, Training of SOM, Topology function, common distance and topology functions, relationship between learning rates and neighbourhoods, applications of SOMs.
  • B. Introduction to Adaptive Resonance Theory, Task and structure of an ART Network, Learning process of an ART Network- top down and bottom up learning, Extensions- ART2, ART3.
  • C. Introduction to Hobbfield Network, Associative Network (Homogenous & Heterogeneous), Introduction to Restricted Boltzman Machine.

📚 Lectures

Theory lecture materials will be uploaded unit-wise. Each unit will include: