Skip to content

I3D activity recognition model #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
natlouis merged 2 commits into from
Oct 25, 2019
Merged

I3D activity recognition model #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d2d4069
Add i3d model and weights links and train/test scripts
ehofesmann Oct 25, 2019
ba7c110
Ignore .pt files
ehofesmann Oct 25, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ runs/*
models/HGC3D
*.json
pbs/*
*.pt
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Check out our [wiki!](https://github.com/MichiganCOG/ViP/wiki)
### Recognition
| Model Architecture | Dataset | ViP Accuracy (%) |
|:--------------------:|:------------------:|:---------------------:|
| I3D | HMDB51 (Split 1) | 72.75 |
| C3D | HMDB51 (Split 1) | 50.14 ± 0.777 |
| C3D | UCF101 (Split 1) | 80.40 ± 0.399 |

Expand Down Expand Up @@ -43,6 +44,7 @@ Check out our [wiki!](https://github.com/MichiganCOG/ViP/wiki)
| Model | Task(s) |
|:------------------------------------------------:|:--------------------:|
|[C3D](https://github.com/jfzhang95/pytorch-video-recognition/blob/master/network/C3D_model.py) | Activity Recognition |
|[I3D](https://github.com/piergiaj/pytorch-i3d) | Activity Recognition |
|[SSD300](https://github.com/amdegroot/ssd.pytorch) | Object Detection |

## Requirements
Expand Down
27 changes: 27 additions & 0 deletions models/i3d/config_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Preprocessing
clip_length: 64 # Number of frames within a clip
clip_offset: 0 # Frame offset between beginning of video and clip (1st clip only)
clip_stride: 0 # Frame offset between successive frames
crop_shape: [224,224] # (Height, Width) of frame
crop_type: Center # Type of cropping operation (Random, Central and None)
final_shape: [224,224] # (Height, Width) of input to be given to CNN
num_clips: -1 # Number clips to be generated from a video (<0: uniform sampling, 0: Divide entire video into clips, >0: Defines number of clips)
random_offset: 0 # Boolean switch to generate a clip length sized clip from a video
resize_shape: [230,250] # (Height, Width) to resize original data
subtract_mean: [123,117,104] # Subtract mean (R,G,B) from all frames during preprocessing

# Experiment Setup
acc_metric: Accuracy # Accuracy metric
batch_size: 1 # Numbers of videos in a mini-batch
dataset: HMDB51 # Name of dataset
exp: I3D # Experiment name
json_path: /z/dat/HMDB51/ # Path to the json file for the given dataset
labels: 51 # Number of total classes in the dataset
load_type: train_val # Environment selection, to include only training/training and validation/testing dataset
model: I3D # Name of model to be loaded
num_workers: 5 # Number of CPU worker used to load data
preprocess: default # String argument to select preprocessing type
pretrained: 'weights/i3d_rgb_imagenet_then_HMDB51_30epochs.pkl' # Load pretrained network
save_dir: './results' # Path to results directory
seed: 999 # Seed for reproducibility
loss_type: M_XENTROPY # Loss function
37 changes: 37 additions & 0 deletions models/i3d/config_train.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Preprocessing
clip_length: 64 # Number of frames within a clip
clip_offset: 0 # Frame offset between beginning of video and clip (1st clip only)
clip_stride: 1 # Frame offset between successive frames
crop_shape: [224,224] # (Height, Width) of frame
crop_type: Center # Type of cropping operation (Random, Central and None)
final_shape: [224,224] # (Height, Width) of input to be given to CNN
num_clips: -1 # Number clips to be generated from a video (<0: uniform sampling, 0: Divide entire video into clips, >0: Defines number of clips)
random_offset: 0 # Boolean switch to generate a clip length sized clip from a video
resize_shape: [230,250] # (Height, Width) to resize original data
subtract_mean: [123,117,104] # Subtract mean (R,G,B) from all frames during preprocessing

# Experiment Setup
acc_metric: Accuracy # Accuracy metric
batch_size: 5 # Numbers of videos in a mini-batch
pseudo_batch_loop: 10 # Pseudo-batch size multiplier to mimic large minibatches
dataset: HMDB51 # Name of dataset
epoch: 30 # Total number of epochs
exp: I3D # Experiment name
gamma: 0.1 # Multiplier with which to change learning rate
json_path: /z/dat/HMDB51/ # Path to the json file for the given dataset
labels: 51 # Number of total classes in the dataset
load_type: train # Environment selection, to include only training/training and validation/testing dataset
loss_type: M_XENTROPY # Loss function
lr: 0.01 # Learning rate
milestones: [10, 20] # Epoch values to change learning rate
model: I3D # Name of model to be loaded
momentum: 0.9 # Momentum value in optimizer
num_workers: 5 # Number of CPU worker used to load data
opt: sgd # Name of optimizer
preprocess: default # String argument to select preprocessing type
pretrained: 1 # Load pretrained network
rerun: 1 # Number of trials to repeat an experiment
save_dir: './results' # Path to results directory
seed: 999 # Seed for reproducibility
weight_decay: 0.0005 # Weight decay
grad_max_norm: 100
Loading