NeuroIncept Decoder

This project focuses on reconstructing speech from EEG signals using deep learning techniques. It processes audio recordings from multiple participants, extracts features, reconstructs audio, and visualizes results.

Technology Stack

Project Structure

• src/: Source code directory
  • feature_extractor.py: EEG and audio data processing
  • audio_constructor.py: Speech reconstruction
  • config.py: Project configuration
  • models.py: Model definitions
  • visualization.py: Result visualization
• main.py: Main execution script
• SingleWordProductionDutch-iBIDS/: Dataset directory
• features/: Extracted features storage
• results/: Reconstruction results and visualizations

Preprocessing Pipeline

Model Architecture

Setup and Usage

1. Clone the repository
2. Install dependencies:

   pip install -r requirements.txt

3. Configure the project in src/config.py
4. Run the main script:

   python main.py

Configuration

Key parameters in src/config.py:

• num_jobs: Number of parallel jobs for processing (default: 20)
• extract_features: Whether to run feature extraction (True or False)
• construct: Whether to run the audio reconstruction process (True or False)
• visualization: Whether to generate visualizations (True or False)
• current_dir: The current working directory
• dataset_dir: Directory containing the dataset (SingleWordProductionDutch-iBIDS)
• features_dir: Directory where extracted features are stored
• results_dir: Directory where the results (e.g., reconstructions, plots) will be saved
• no_of_mel_spectrograms: The number of mel spectral bins (default: 128)
• epochs: Number of epochs for model training (default: 100)
• batch_size: Batch size for model training (default: 128)
• num_folds: Number of folds for cross-validation (default: 10)

Visualization

When enabled, the project generates:

• Correlation plots: Shows the bar plot for correlation between the predicted spectrograms by NeuroIncept Decoder Model and original spectrograms for all subjects
• STGI (Short-Time Geometric Image) plots: Displays Bar Plot of STGI metric for all Subjects
• Training history plots: Plots the history of the model's training process, Validation loss
• Spectrogram comparisons: Spectrograms generated by various model architectures

Dataset

The project uses the SingleWordProductionDutch-iBIDS dataset. Place it in the directory specified by dataset_dir in config.py.