diff --git a/.gitmodules b/.gitmodules
new file mode 100644
index 0000000..7b01579
--- /dev/null
+++ b/.gitmodules
@@ -0,0 +1,6 @@
+[submodule "Pretrain/UniFormerV2"]
+ path = Pretrain/UniFormerV2
+ url = https://github.com/OpenGVLab/UniFormerV2.git
+[submodule "Downstream/Ego-Tasks"]
+ path = Downstream/Ego-Tasks
+ url = https://github.com/OpenGVLab/ego4d-eccv2022-solutions.git
diff --git a/Data/InternVid/README.md b/Data/InternVid/README.md
new file mode 100644
index 0000000..f924062
--- /dev/null
+++ b/Data/InternVid/README.md
@@ -0,0 +1,60 @@
+# InternVid \[[Paper](https://arxiv.org/pdf/2307.06942.pdf)\]
+
+[![Dataset meta](https://img.shields.io/badge/%F0%9F%A4%97%20InternVid-Dataset-blue)](https://huggingface.co/datasets/OpenGVLab/InternVid) | [![Model Checkpoint](https://img.shields.io/badge/%F0%9F%A4%97%20ViCLIP-Model-purple)](https://huggingface.co/OpenGVLab/ViCLIP)
+
+# :fire: News
+We are excited to announce the partial release of a large-scale video-text dataset aimed at facilitating multimodal understanding and generation. As part of this release, we are making available a [subset](https://huggingface.co/datasets/OpenGVLab/InternVid) of the dataset, which comprises 10 million video clips. Additionally, we have provided a [ViCLIP](https://huggingface.co/OpenGVLab/ViCLIP) model trained on this subset, using the ViT-L architecture. It achieves SOTA zero-shot action recognition performance on Kinetics.
+
+We give a step-by-step instructions and clarify the process of accessing and utilizing ViClip in [demo.ipynb](https://github.com/OpenGVLab/InternVideo/blob/main/Data/InternVid/demo.ipynb).
+
+Stay tuned for updates!
+
+# Introduction
+
+**Data**
+
+We collected videos from 16 popular categories with varying percentages. We ensured diversity by selecting videos from countries with different languages instead of relying on a dominant language environment. The countries we sampled from include the UK, USA, Australia, Japan, Korea, China, Russia, and France, among others. In terms of duration, every video lasts 351.9s on average. Almost half (49%) of the videos are five minutes or less, while a quarter (26%) fall between five and ten minutes. Only 8% of the videos are over 20 minutes long. Among the curated videos, 85% were high-resolution (720P), while the remaining 15% had lower resolutions ranging from 360P to 720P. Although the lower-resolution videos may not perform as well as the high-resolution ones in content generation tasks, they can still be useful in video-language representation learning, provided that they have appropriate captions.
+
+![b469e00b43d46a6b3f89899483abcf6](https://github.com/OpenGVLab/InternVideo/assets/43169235/7d6aca7d-362a-425d-9ef2-ec0189491b52)
+
+InternVid exhibits diverse clip durations and caption lengths in the segmented clip level. The aesthetic scores and clip-caption similarities are distributed uniformly. The majority of clips are 0-10 seconds in length, accounting for 85% of all clips. Approximately half of the clips have captions with 10-20 words, while one-third of the clip captions have fewer than 10 words. About 11% of clips have long captions with more than 20 words.
+
+![429af4993adb77478c000c865ae5a1b](https://github.com/OpenGVLab/InternVideo/assets/43169235/f64588c3-81e8-43de-b771-46500474d2ff)
+
+**ViCLIP: a simple video CLIP for transferrable video-text representation**
+
+Built upon CLIP , we make a simple video-text pretraining baseline ViCLIP. It consists of a video encoder (ViT) and a text encoder, as given below. Both modules are initialized from the corresponding CLIP components. We update the native attention in the video encoder to spatiotemporal attention while maintaining other design elements. For efficient learning, we apply masking to videos in pre-training.
+
+
+
+
+# Data & Model Zoo
+
+### Pretrained Data & Model
+
+
+| Model | Training Data | Descriptions |
+| :-----------------: | :----------------------: | :---------------------------------------------------------------------------------------------------: |
+| ViCLIP-L-14 \[[HuggingFace](https://huggingface.co/OpenGVLab/ViCLIP) \| [Aliyun](https://pjlab-gvm-data.oss-cn-shanghai.aliyuncs.com/internvideo/viclip/ViClip-InternVid-10M-FLT.pth )\] | InternVid-10M-FLT \[[HuggingFace](https://huggingface.co/datasets/OpenGVLab/InternVid) \| [OpenDataLab](https://opendatalab.com/shepshep/InternVid)\] | |
+
+
+
+## Citation
+
+If you find this work useful for your research, please consider citing InternVid. Your acknowledgement would greatly help us in continuing to contribute resources to the research community.
+
+```
+@article{wang2023internvid,
+ title={InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation},
+ author={Wang, Yi and He, Yinan and Li, Yizhuo and Li, Kunchang and Yu, Jiashuo and Ma, Xin and Chen, Xinyuan and Wang, Yaohui and Luo, Ping and Liu, Ziwei and Wang, Yali and Wang, Limin and Qiao, Yu},
+ journal={arXiv preprint arXiv:2307.06942},
+ year={2023}
+}
+
+@article{wang2022internvideo,
+ title={InternVideo: General Video Foundation Models via Generative and Discriminative Learning},
+ author={Wang, Yi and Li, Kunchang and Li, Yizhuo and He, Yinan and Huang, Bingkun and Zhao, Zhiyu and Zhang, Hongjie and Xu, Jilan and Liu, Yi and Wang, Zun and Xing, Sen and Chen, Guo and Pan, Junting and Yu, Jiashuo and Wang, Yali and Wang, Limin and Qiao, Yu},
+ journal={arXiv preprint arXiv:2212.03191},
+ year={2022}
+}
+```
diff --git a/Data/InternVid/demo.ipynb b/Data/InternVid/demo.ipynb
new file mode 100644
index 0000000..586abfc
--- /dev/null
+++ b/Data/InternVid/demo.ipynb
@@ -0,0 +1,102 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "f86bc499",
+ "metadata": {},
+ "source": [
+ "## download ViCILP weights and put its pth file in viclip folder. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e7a90379-d9ee-45d9-9073-7ed5132fa6b1",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import cv2\n",
+ "\n",
+ "from viclip import retrieve_text, _frame_from_video"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "a425a5da-ceaf-4b89-9845-c8ba576902d8",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "video = cv2.VideoCapture('example1.mp4')\n",
+ "frames = [x for x in _frame_from_video(video)]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "id": "3fb7397a-02ef-41b5-9ffe-f2363b277778",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "text: A man in a gray sweater plays fetch with his dog in the snowy yard, throwing a toy and watching it run. ~ prob: 0.8264\n",
+ "text: A playful dog and its owner wrestle in the snowy yard, chasing each other with joyous abandon. ~ prob: 0.1587\n",
+ "text: A pet dog excitedly runs through the snowy yard, chasing a toy thrown by its owner. ~ prob: 0.0141\n",
+ "text: A person dressed in a blue jacket shovels the snow-covered pavement outside their house. ~ prob: 0.0006\n",
+ "text: A playful dog slides down a snowy hill, wagging its tail with delight. ~ prob: 0.0002\n"
+ ]
+ }
+ ],
+ "source": [
+ "text_candidates = [\"A playful dog and its owner wrestle in the snowy yard, chasing each other with joyous abandon.\",\n",
+ " \"A man in a gray coat walks through the snowy landscape, pulling a sleigh loaded with toys.\",\n",
+ " \"A person dressed in a blue jacket shovels the snow-covered pavement outside their house.\",\n",
+ " \"A pet dog excitedly runs through the snowy yard, chasing a toy thrown by its owner.\",\n",
+ " \"A person stands on the snowy floor, pushing a sled loaded with blankets, preparing for a fun-filled ride.\",\n",
+ " \"A man in a gray hat and coat walks through the snowy yard, carefully navigating around the trees.\",\n",
+ " \"A playful dog slides down a snowy hill, wagging its tail with delight.\",\n",
+ " \"A person in a blue jacket walks their pet on a leash, enjoying a peaceful winter walk among the trees.\",\n",
+ " \"A man in a gray sweater plays fetch with his dog in the snowy yard, throwing a toy and watching it run.\",\n",
+ " \"A person bundled up in a blanket walks through the snowy landscape, enjoying the serene winter scenery.\"]\n",
+ "\n",
+ "texts, probs = retrieve_text(frames, text_candidates, name='viclip', topk=5)\n",
+ "\n",
+ "for t, p in zip(texts, probs):\n",
+ " print(f'text: {t} ~ prob: {p:.4f}')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a2969ba6-19d0-4893-b071-b82fa046c312",
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.9.12"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/Data/InternVid/example1.mp4 b/Data/InternVid/example1.mp4
new file mode 100644
index 0000000..815f841
Binary files /dev/null and b/Data/InternVid/example1.mp4 differ
diff --git a/Data/InternVid/viclip/README.md b/Data/InternVid/viclip/README.md
new file mode 100644
index 0000000..32897cd
--- /dev/null
+++ b/Data/InternVid/viclip/README.md
@@ -0,0 +1,3 @@
+---
+license: mit
+---
diff --git a/Data/InternVid/viclip/__init__.py b/Data/InternVid/viclip/__init__.py
new file mode 100644
index 0000000..389eb2c
--- /dev/null
+++ b/Data/InternVid/viclip/__init__.py
@@ -0,0 +1,71 @@
+from .simple_tokenizer import SimpleTokenizer as _Tokenizer
+from .viclip import ViCLIP
+import torch
+import numpy as np
+import cv2
+
+clip_candidates = {'viclip':None, 'clip':None}
+
+def get_clip(name='viclip'):
+ global clip_candidates
+ m = clip_candidates[name]
+ if m is None:
+ if name == 'viclip':
+ tokenizer = _Tokenizer()
+ vclip = ViCLIP(tokenizer)
+ # m = vclip
+ m = (vclip, tokenizer)
+ else:
+ raise Exception('the target clip model is not found.')
+
+ return m
+
+def get_text_feat_dict(texts, clip, tokenizer, text_feat_d={}):
+ for t in texts:
+ feat = clip.get_text_features(t, tokenizer, text_feat_d)
+ text_feat_d[t] = feat
+ return text_feat_d
+
+def get_vid_feat(frames, clip):
+ return clip.get_vid_features(frames)
+
+def _frame_from_video(video):
+ while video.isOpened():
+ success, frame = video.read()
+ if success:
+ yield frame
+ else:
+ break
+
+v_mean = np.array([0.485, 0.456, 0.406]).reshape(1,1,3)
+v_std = np.array([0.229, 0.224, 0.225]).reshape(1,1,3)
+def normalize(data):
+ return (data/255.0-v_mean)/v_std
+
+def frames2tensor(vid_list, fnum=8, target_size=(224, 224), device=torch.device('cuda')):
+ assert(len(vid_list) >= fnum)
+ step = len(vid_list) // fnum
+ vid_list = vid_list[::step][:fnum]
+ vid_list = [cv2.resize(x[:,:,::-1], target_size) for x in vid_list]
+ vid_tube = [np.expand_dims(normalize(x), axis=(0, 1)) for x in vid_list]
+ vid_tube = np.concatenate(vid_tube, axis=1)
+ vid_tube = np.transpose(vid_tube, (0, 1, 4, 2, 3))
+ vid_tube = torch.from_numpy(vid_tube).to(device, non_blocking=True).float()
+ return vid_tube
+
+def retrieve_text(frames, texts, name='viclip', topk=5, device=torch.device('cuda')):
+ clip, tokenizer = get_clip(name)
+ clip = clip.to(device)
+ frames_tensor = frames2tensor(frames, device=device)
+ vid_feat = get_vid_feat(frames_tensor, clip)
+
+ text_feat_d = {}
+ text_feat_d = get_text_feat_dict(texts, clip, tokenizer, text_feat_d)
+ text_feats = [text_feat_d[t] for t in texts]
+ text_feats_tensor = torch.cat(text_feats, 0)
+
+ probs, idxs = clip.get_predict_label(vid_feat, text_feats_tensor, top=topk)
+
+ ret_texts = [texts[i] for i in idxs.numpy()[0].tolist()]
+ return ret_texts, probs.numpy()[0]
+
diff --git a/Data/InternVid/viclip/bpe_simple_vocab_16e6.txt.gz b/Data/InternVid/viclip/bpe_simple_vocab_16e6.txt.gz
new file mode 100644
index 0000000..7b5088a
Binary files /dev/null and b/Data/InternVid/viclip/bpe_simple_vocab_16e6.txt.gz differ
diff --git a/Data/InternVid/viclip/simple_tokenizer.py b/Data/InternVid/viclip/simple_tokenizer.py
new file mode 100644
index 0000000..0f2feb7
--- /dev/null
+++ b/Data/InternVid/viclip/simple_tokenizer.py
@@ -0,0 +1,135 @@
+import gzip
+import html
+import os
+from functools import lru_cache
+
+import ftfy
+import regex as re
+
+
+@lru_cache()
+def default_bpe():
+ return os.path.join(os.path.dirname(os.path.abspath(__file__)), "bpe_simple_vocab_16e6.txt.gz")
+# @lru_cache()
+# def default_bpe():
+# return "bpe_simple_vocab_16e6.txt.gz"
+
+
+@lru_cache()
+def bytes_to_unicode():
+ """
+ Returns list of utf-8 byte and a corresponding list of unicode strings.
+ The reversible bpe codes work on unicode strings.
+ This means you need a large # of unicode characters in your vocab if you want to avoid UNKs.
+ When you're at something like a 10B token dataset you end up needing around 5K for decent coverage.
+ This is a signficant percentage of your normal, say, 32K bpe vocab.
+ To avoid that, we want lookup tables between utf-8 bytes and unicode strings.
+ And avoids mapping to whitespace/control characters the bpe code barfs on.
+ """
+ bs = list(range(ord("!"), ord("~")+1))+list(range(ord("¡"), ord("¬")+1))+list(range(ord("®"), ord("ÿ")+1))
+ cs = bs[:]
+ n = 0
+ for b in range(2**8):
+ if b not in bs:
+ bs.append(b)
+ cs.append(2**8+n)
+ n += 1
+ cs = [chr(n) for n in cs]
+ return dict(zip(bs, cs))
+
+
+def get_pairs(word):
+ """Return set of symbol pairs in a word.
+ Word is represented as tuple of symbols (symbols being variable-length strings).
+ """
+ pairs = set()
+ prev_char = word[0]
+ for char in word[1:]:
+ pairs.add((prev_char, char))
+ prev_char = char
+ return pairs
+
+
+def basic_clean(text):
+ text = ftfy.fix_text(text)
+ text = html.unescape(html.unescape(text))
+ return text.strip()
+
+
+def whitespace_clean(text):
+ text = re.sub(r'\s+', ' ', text)
+ text = text.strip()
+ return text
+
+
+class SimpleTokenizer(object):
+ def __init__(self, bpe_path: str = default_bpe()):
+ self.byte_encoder = bytes_to_unicode()
+ self.byte_decoder = {v: k for k, v in self.byte_encoder.items()}
+ merges = gzip.open(bpe_path).read().decode("utf-8").split('\n')
+ merges = merges[1:49152-256-2+1]
+ merges = [tuple(merge.split()) for merge in merges]
+ vocab = list(bytes_to_unicode().values())
+ vocab = vocab + [v+'' for v in vocab]
+ for merge in merges:
+ vocab.append(''.join(merge))
+ vocab.extend(['<|startoftext|>', '<|endoftext|>'])
+ self.encoder = dict(zip(vocab, range(len(vocab))))
+ self.decoder = {v: k for k, v in self.encoder.items()}
+ self.bpe_ranks = dict(zip(merges, range(len(merges))))
+ self.cache = {'<|startoftext|>': '<|startoftext|>', '<|endoftext|>': '<|endoftext|>'}
+ self.pat = re.compile(r"""<\|startoftext\|>|<\|endoftext\|>|'s|'t|'re|'ve|'m|'ll|'d|[\p{L}]+|[\p{N}]|[^\s\p{L}\p{N}]+""", re.IGNORECASE)
+
+ def bpe(self, token):
+ if token in self.cache:
+ return self.cache[token]
+ word = tuple(token[:-1]) + ( token[-1] + '',)
+ pairs = get_pairs(word)
+
+ if not pairs:
+ return token+''
+
+ while True:
+ bigram = min(pairs, key = lambda pair: self.bpe_ranks.get(pair, float('inf')))
+ if bigram not in self.bpe_ranks:
+ break
+ first, second = bigram
+ new_word = []
+ i = 0
+ while i < len(word):
+ try:
+ j = word.index(first, i)
+ new_word.extend(word[i:j])
+ i = j
+ except:
+ new_word.extend(word[i:])
+ break
+
+ if word[i] == first and i < len(word)-1 and word[i+1] == second:
+ new_word.append(first+second)
+ i += 2
+ else:
+ new_word.append(word[i])
+ i += 1
+ new_word = tuple(new_word)
+ word = new_word
+ if len(word) == 1:
+ break
+ else:
+ pairs = get_pairs(word)
+ word = ' '.join(word)
+ self.cache[token] = word
+ return word
+
+ def encode(self, text):
+ bpe_tokens = []
+ text = whitespace_clean(basic_clean(text)).lower()
+ for token in re.findall(self.pat, text):
+ token = ''.join(self.byte_encoder[b] for b in token.encode('utf-8'))
+ bpe_tokens.extend(self.encoder[bpe_token] for bpe_token in self.bpe(token).split(' '))
+ return bpe_tokens
+
+ def decode(self, tokens):
+ text = ''.join([self.decoder[token] for token in tokens])
+ text = bytearray([self.byte_decoder[c] for c in text]).decode('utf-8', errors="replace").replace('', ' ')
+ return text
diff --git a/Data/InternVid/viclip/viclip.py b/Data/InternVid/viclip/viclip.py
new file mode 100644
index 0000000..02ed605
--- /dev/null
+++ b/Data/InternVid/viclip/viclip.py
@@ -0,0 +1,229 @@
+import os
+import logging
+
+import torch
+from einops import rearrange
+from torch import nn
+import math
+
+# from .criterions import VTC_VTM_Loss
+from .simple_tokenizer import SimpleTokenizer as _Tokenizer
+from .viclip_vision import clip_joint_l14
+from .viclip_text import clip_text_l14
+
+logger = logging.getLogger(__name__)
+
+
+class ViCLIP(nn.Module):
+ """docstring for ViCLIP"""
+
+ def __init__(self, tokenizer=None, pretrain=os.path.join(os.path.dirname(os.path.abspath(__file__)), "ViClip-InternVid-10M-FLT.pth"), freeze_text=True):
+ super(ViCLIP, self).__init__()
+ if tokenizer:
+ self.tokenizer = tokenizer
+ else:
+ self.tokenizer = _Tokenizer()
+ self.max_txt_l = 32
+
+ self.vision_encoder_name = 'vit_l14'
+
+ self.vision_encoder_pretrained = False
+ self.inputs_image_res = 224
+ self.vision_encoder_kernel_size = 1
+ self.vision_encoder_center = True
+ self.video_input_num_frames = 8
+ self.vision_encoder_drop_path_rate = 0.1
+ self.vision_encoder_checkpoint_num = 24
+ self.is_pretrain = pretrain
+ self.vision_width = 1024
+ self.text_width = 768
+ self.embed_dim = 768
+ self.masking_prob = 0.9
+
+ self.text_encoder_name = 'vit_l14'
+ self.text_encoder_pretrained = False#'bert-base-uncased'
+ self.text_encoder_d_model = 768
+
+ self.text_encoder_vocab_size = 49408
+
+
+ # create modules.
+ self.vision_encoder = self.build_vision_encoder()
+ self.text_encoder = self.build_text_encoder()
+
+ self.temp = nn.parameter.Parameter(torch.ones([]) * 1 / 100.0)
+ self.temp_min = 1 / 100.0
+
+ if pretrain:
+ logger.info(f"Load pretrained weights from {pretrain}")
+ state_dict = torch.load(pretrain, map_location='cpu')['model']
+ self.load_state_dict(state_dict)
+
+ # Freeze weights
+ if freeze_text:
+ self.freeze_text()
+
+
+
+ def freeze_text(self):
+ """freeze text encoder"""
+ for p in self.text_encoder.parameters():
+ p.requires_grad = False
+
+ def no_weight_decay(self):
+ ret = {"temp"}
+ ret.update(
+ {"vision_encoder." + k for k in self.vision_encoder.no_weight_decay()}
+ )
+ ret.update(
+ {"text_encoder." + k for k in self.text_encoder.no_weight_decay()}
+ )
+
+ return ret
+
+ def forward(self, image, text, raw_text, idx, log_generation=None, return_sims=False):
+ """forward and calculate loss.
+
+ Args:
+ image (torch.Tensor): The input images. Shape: [B,T,C,H,W].
+ text (dict): TODO
+ idx (torch.Tensor): TODO
+
+ Returns: TODO
+
+ """
+ self.clip_contrastive_temperature()
+
+ vision_embeds = self.encode_vision(image)
+ text_embeds = self.encode_text(raw_text)
+ if return_sims:
+ sims = torch.nn.functional.normalize(vision_embeds, dim=-1) @ \
+ torch.nn.functional.normalize(text_embeds, dim=-1).transpose(0, 1)
+ return sims
+
+ # calculate loss
+
+ ## VTC loss
+ loss_vtc = self.clip_loss.vtc_loss(
+ vision_embeds, text_embeds, idx, self.temp, all_gather=True
+ )
+
+ return dict(
+ loss_vtc=loss_vtc,
+ )
+
+ def encode_vision(self, image, test=False):
+ """encode image / videos as features.
+
+ Args:
+ image (torch.Tensor): The input images.
+ test (bool): Whether testing.
+
+ Returns: tuple.
+ - vision_embeds (torch.Tensor): The features of all patches. Shape: [B,T,L,C].
+ - pooled_vision_embeds (torch.Tensor): The pooled features. Shape: [B,T,C].
+
+ """
+ if image.ndim == 5:
+ image = image.permute(0, 2, 1, 3, 4).contiguous()
+ else:
+ image = image.unsqueeze(2)
+
+ if not test and self.masking_prob > 0.0:
+ return self.vision_encoder(
+ image, masking_prob=self.masking_prob
+ )
+
+ return self.vision_encoder(image)
+
+ def encode_text(self, text):
+ """encode text.
+ Args:
+ text (dict): The output of huggingface's `PreTrainedTokenizer`. contains keys:
+ - input_ids (torch.Tensor): Token ids to be fed to a model. Shape: [B,L].
+ - attention_mask (torch.Tensor): The mask indicate padded tokens. Shape: [B,L]. 0 is padded token.
+ - other keys refer to "https://huggingface.co/docs/transformers/v4.21.2/en/main_classes/tokenizer#transformers.PreTrainedTokenizer.__call__".
+ Returns: tuple.
+ - text_embeds (torch.Tensor): The features of all tokens. Shape: [B,L,C].
+ - pooled_text_embeds (torch.Tensor): The pooled features. Shape: [B,C].
+
+ """
+ device = next(self.text_encoder.parameters()).device
+ text = self.text_encoder.tokenize(
+ text, context_length=self.max_txt_l
+ ).to(device)
+ text_embeds = self.text_encoder(text)
+ return text_embeds
+
+ @torch.no_grad()
+ def clip_contrastive_temperature(self, min_val=0.001, max_val=0.5):
+ """Seems only used during pre-training"""
+ self.temp.clamp_(min=self.temp_min)
+
+ def build_vision_encoder(self):
+ """build vision encoder
+ Returns: (vision_encoder, vision_layernorm). Each is a `nn.Module`.
+
+ """
+ encoder_name = self.vision_encoder_name
+ if encoder_name != "vit_l14":
+ raise ValueError(f"Not implemented: {encoder_name}")
+ vision_encoder = clip_joint_l14(
+ pretrained=self.vision_encoder_pretrained,
+ input_resolution=self.inputs_image_res,
+ kernel_size=self.vision_encoder_kernel_size,
+ center=self.vision_encoder_center,
+ num_frames=self.video_input_num_frames,
+ drop_path=self.vision_encoder_drop_path_rate,
+ checkpoint_num=self.vision_encoder_checkpoint_num,
+ )
+ return vision_encoder
+
+ def build_text_encoder(self):
+ """build text_encoder and possiblly video-to-text multimodal fusion encoder.
+ Returns: nn.Module. The text encoder
+
+ """
+ encoder_name = self.text_encoder_name
+ if encoder_name != "vit_l14":
+ raise ValueError(f"Not implemented: {encoder_name}")
+ text_encoder = clip_text_l14(
+ pretrained=self.text_encoder_pretrained,
+ embed_dim=self.text_encoder_d_model,
+ context_length=self.max_txt_l,
+ vocab_size=self.text_encoder_vocab_size,
+ checkpoint_num=0,
+ )
+
+ return text_encoder
+
+ def get_text_encoder(self):
+ """get text encoder, used for text and cross-modal encoding"""
+ encoder = self.text_encoder
+ return encoder.bert if hasattr(encoder, "bert") else encoder
+
+ def get_text_features(self, input_text, tokenizer, text_feature_dict={}):
+ if input_text in text_feature_dict:
+ return text_feature_dict[input_text]
+ text_template= f"{input_text}"
+ with torch.no_grad():
+ # text_token = tokenizer.encode(text_template).cuda()
+ text_features = self.encode_text(text_template).float()
+ text_features /= text_features.norm(dim=-1, keepdim=True)
+ text_feature_dict[input_text] = text_features
+ return text_features
+
+ def get_vid_features(self, input_frames):
+ with torch.no_grad():
+ clip_feat = self.encode_vision(input_frames,test=True).float()
+ clip_feat /= clip_feat.norm(dim=-1, keepdim=True)
+ return clip_feat
+
+ def get_predict_label(self, clip_feature, text_feats_tensor, top=5):
+ label_probs = (100.0 * clip_feature @ text_feats_tensor.T).softmax(dim=-1)
+ top_probs, top_labels = label_probs.cpu().topk(top, dim=-1)
+ return top_probs, top_labels
+
+
+if __name__ =="__main__":
+ tokenizer = _Tokenizer()
diff --git a/Data/InternVid/viclip/viclip_text.py b/Data/InternVid/viclip/viclip_text.py
new file mode 100644
index 0000000..6d9a959
--- /dev/null
+++ b/Data/InternVid/viclip/viclip_text.py
@@ -0,0 +1,271 @@
+import os
+import logging
+from collections import OrderedDict
+from pkg_resources import packaging
+from .simple_tokenizer import SimpleTokenizer as _Tokenizer
+
+import numpy as np
+import torch
+import torch.nn.functional as F
+from torch import nn
+import torch.utils.checkpoint as checkpoint
+import functools
+
+logger = logging.getLogger(__name__)
+
+
+# On P1, model extracted from https://huggingface.co/laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K
+MODEL_PATH = 'https://huggingface.co/laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K'
+_MODELS = {
+ "ViT-L/14": os.path.join(MODEL_PATH, "vit_l14_text.pth"),
+}
+
+
+class LayerNorm(nn.LayerNorm):
+ """Subclass torch's LayerNorm to handle fp16."""
+
+ def forward(self, x: torch.Tensor):
+ orig_type = x.dtype
+ ret = super().forward(x.type(torch.float32))
+ return ret.type(orig_type)
+
+
+class QuickGELU(nn.Module):
+ def forward(self, x: torch.Tensor):
+ return x * torch.sigmoid(1.702 * x)
+
+
+class ResidualAttentionBlock(nn.Module):
+ def __init__(self, d_model: int, n_head: int, attn_mask: torch.Tensor = None):
+ super().__init__()
+
+ self.attn = nn.MultiheadAttention(d_model, n_head)
+ self.ln_1 = LayerNorm(d_model)
+ self.mlp = nn.Sequential(OrderedDict([
+ ("c_fc", nn.Linear(d_model, d_model * 4)),
+ ("gelu", QuickGELU()),
+ ("c_proj", nn.Linear(d_model * 4, d_model))
+ ]))
+ self.ln_2 = LayerNorm(d_model)
+ self.attn_mask = attn_mask
+
+ def attention(self, x: torch.Tensor):
+ self.attn_mask = self.attn_mask.to(dtype=x.dtype, device=x.device) if self.attn_mask is not None else None
+ return self.attn(x, x, x, need_weights=False, attn_mask=self.attn_mask)[0]
+
+ def forward(self, x: torch.Tensor):
+ x = x + self.attention(self.ln_1(x))
+ x = x + self.mlp(self.ln_2(x))
+ return x
+
+
+class Transformer(nn.Module):
+ def __init__(self, width: int, layers: int, heads: int, attn_mask: torch.Tensor = None,
+ checkpoint_num: int = 0):
+ super().__init__()
+ self.width = width
+ self.layers = layers
+ self.resblocks = nn.Sequential(*[ResidualAttentionBlock(width, heads, attn_mask) for _ in range(layers)])
+
+ self.checkpoint_num = checkpoint_num
+
+ def forward(self, x: torch.Tensor):
+ if self.checkpoint_num > 0:
+ segments = min(self.checkpoint_num, len(self.resblocks))
+ return checkpoint.checkpoint_sequential(self.resblocks, segments, x)
+ else:
+ return self.resblocks(x)
+
+
+class CLIP_TEXT(nn.Module):
+ def __init__(
+ self,
+ embed_dim: int,
+ context_length: int,
+ vocab_size: int,
+ transformer_width: int,
+ transformer_heads: int,
+ transformer_layers: int,
+ checkpoint_num: int,
+ ):
+ super().__init__()
+
+ self.context_length = context_length
+ self._tokenizer = _Tokenizer()
+
+ self.transformer = Transformer(
+ width=transformer_width,
+ layers=transformer_layers,
+ heads=transformer_heads,
+ attn_mask=self.build_attention_mask(),
+ checkpoint_num=checkpoint_num,
+ )
+
+ self.vocab_size = vocab_size
+ self.token_embedding = nn.Embedding(vocab_size, transformer_width)
+ self.positional_embedding = nn.Parameter(torch.empty(self.context_length, transformer_width))
+ self.ln_final = LayerNorm(transformer_width)
+
+ self.text_projection = nn.Parameter(torch.empty(transformer_width, embed_dim))
+
+ def no_weight_decay(self):
+ return {'token_embedding', 'positional_embedding'}
+
+ @functools.lru_cache(maxsize=None)
+ def build_attention_mask(self):
+ # lazily create causal attention mask, with full attention between the vision tokens
+ # pytorch uses additive attention mask; fill with -inf
+ mask = torch.empty(self.context_length, self.context_length)
+ mask.fill_(float("-inf"))
+ mask.triu_(1) # zero out the lower diagonal
+ return mask
+
+ def tokenize(self, texts, context_length=77, truncate=True):
+ """
+ Returns the tokenized representation of given input string(s)
+ Parameters
+ ----------
+ texts : Union[str, List[str]]
+ An input string or a list of input strings to tokenize
+ context_length : int
+ The context length to use; all CLIP models use 77 as the context length
+ truncate: bool
+ Whether to truncate the text in case its encoding is longer than the context length
+ Returns
+ -------
+ A two-dimensional tensor containing the resulting tokens, shape = [number of input strings, context_length].
+ We return LongTensor when torch version is <1.8.0, since older index_select requires indices to be long.
+ """
+ if isinstance(texts, str):
+ texts = [texts]
+
+ sot_token = self._tokenizer.encoder["<|startoftext|>"]
+ eot_token = self._tokenizer.encoder["<|endoftext|>"]
+ all_tokens = [[sot_token] + self._tokenizer.encode(text) + [eot_token] for text in texts]
+ if packaging.version.parse(torch.__version__) < packaging.version.parse("1.8.0"):
+ result = torch.zeros(len(all_tokens), context_length, dtype=torch.long)
+ else:
+ result = torch.zeros(len(all_tokens), context_length, dtype=torch.int)
+
+ for i, tokens in enumerate(all_tokens):
+ if len(tokens) > context_length:
+ if truncate:
+ tokens = tokens[:context_length]
+ tokens[-1] = eot_token
+ else:
+ raise RuntimeError(f"Input {texts[i]} is too long for context length {context_length}")
+ result[i, :len(tokens)] = torch.tensor(tokens)
+
+ return result
+
+ def forward(self, text):
+ x = self.token_embedding(text) # [batch_size, n_ctx, d_model]
+
+ x = x + self.positional_embedding
+ x = x.permute(1, 0, 2) # NLD -> LND
+ x = self.transformer(x)
+ x = x.permute(1, 0, 2) # LND -> NLD
+ x = self.ln_final(x)
+
+ # x.shape = [batch_size, n_ctx, transformer.width]
+ # take features from the eot embedding (eot_token is the highest number in each sequence)
+ x = x[torch.arange(x.shape[0]), text.argmax(dim=-1)] @ self.text_projection
+
+ return x
+
+
+def clip_text_b16(
+ embed_dim=512,
+ context_length=77,
+ vocab_size=49408,
+ transformer_width=512,
+ transformer_heads=8,
+ transformer_layers=12,
+):
+ raise NotImplementedError
+ model = CLIP_TEXT(
+ embed_dim,
+ context_length,
+ vocab_size,
+ transformer_width,
+ transformer_heads,
+ transformer_layers
+ )
+ pretrained = _MODELS["ViT-B/16"]
+ logger.info(f"Load pretrained weights from {pretrained}")
+ state_dict = torch.load(pretrained, map_location='cpu')
+ model.load_state_dict(state_dict, strict=False)
+ return model.eval()
+
+
+def clip_text_l14(
+ embed_dim=768,
+ context_length=77,
+ vocab_size=49408,
+ transformer_width=768,
+ transformer_heads=12,
+ transformer_layers=12,
+ checkpoint_num=0,
+ pretrained=True,
+):
+ model = CLIP_TEXT(
+ embed_dim,
+ context_length,
+ vocab_size,
+ transformer_width,
+ transformer_heads,
+ transformer_layers,
+ checkpoint_num,
+ )
+ if pretrained:
+ if isinstance(pretrained, str) and pretrained != "bert-base-uncased":
+ pretrained = _MODELS[pretrained]
+ else:
+ pretrained = _MODELS["ViT-L/14"]
+ logger.info(f"Load pretrained weights from {pretrained}")
+ state_dict = torch.load(pretrained, map_location='cpu')
+ if context_length != state_dict["positional_embedding"].size(0):
+ # assert context_length < state_dict["positional_embedding"].size(0), "Cannot increase context length."
+ print(f"Resize positional embedding from {state_dict['positional_embedding'].size(0)} to {context_length}")
+ if context_length < state_dict["positional_embedding"].size(0):
+ state_dict["positional_embedding"] = state_dict["positional_embedding"][:context_length]
+ else:
+ state_dict["positional_embedding"] = F.pad(
+ state_dict["positional_embedding"],
+ (0, 0, 0, context_length - state_dict["positional_embedding"].size(0)),
+ value=0,
+ )
+
+ message = model.load_state_dict(state_dict, strict=False)
+ print(f"Load pretrained weights from {pretrained}: {message}")
+ return model.eval()
+
+
+def clip_text_l14_336(
+ embed_dim=768,
+ context_length=77,
+ vocab_size=49408,
+ transformer_width=768,
+ transformer_heads=12,
+ transformer_layers=12,
+):
+ raise NotImplementedError
+ model = CLIP_TEXT(
+ embed_dim,
+ context_length,
+ vocab_size,
+ transformer_width,
+ transformer_heads,
+ transformer_layers
+ )
+ pretrained = _MODELS["ViT-L/14_336"]
+ logger.info(f"Load pretrained weights from {pretrained}")
+ state_dict = torch.load(pretrained, map_location='cpu')
+ model.load_state_dict(state_dict, strict=False)
+ return model.eval()
+
+
+def build_clip(config):
+ model_cls = config.text_encoder.clip_teacher
+ model = eval(model_cls)()
+ return model
diff --git a/Data/InternVid/viclip/viclip_vision.py b/Data/InternVid/viclip/viclip_vision.py
new file mode 100644
index 0000000..1d0598a
--- /dev/null
+++ b/Data/InternVid/viclip/viclip_vision.py
@@ -0,0 +1,354 @@
+#!/usr/bin/env python
+import os
+import logging
+from collections import OrderedDict
+
+import torch
+from torch import nn
+from einops import rearrange
+from timm.models.layers import DropPath
+from timm.models.registry import register_model
+
+import torch.utils.checkpoint as checkpoint
+
+# from models.utils import load_temp_embed_with_mismatch
+
+logger = logging.getLogger(__name__)
+
+def load_temp_embed_with_mismatch(temp_embed_old, temp_embed_new, add_zero=True):
+ """
+ Add/Remove extra temporal_embeddings as needed.
+ https://arxiv.org/abs/2104.00650 shows adding zero paddings works.
+
+ temp_embed_old: (1, num_frames_old, 1, d)
+ temp_embed_new: (1, num_frames_new, 1, d)
+ add_zero: bool, if True, add zero, else, interpolate trained embeddings.
+ """
+ # TODO zero pad
+ num_frms_new = temp_embed_new.shape[1]
+ num_frms_old = temp_embed_old.shape[1]
+ logger.info(f"Load temporal_embeddings, lengths: {num_frms_old}-->{num_frms_new}")
+ if num_frms_new > num_frms_old:
+ if add_zero:
+ temp_embed_new[
+ :, :num_frms_old
+ ] = temp_embed_old # untrained embeddings are zeros.
+ else:
+ temp_embed_new = interpolate_temporal_pos_embed(temp_embed_old, num_frms_new)
+ elif num_frms_new < num_frms_old:
+ temp_embed_new = temp_embed_old[:, :num_frms_new]
+ else: # =
+ temp_embed_new = temp_embed_old
+ return temp_embed_new
+
+
+# On P1, model extracted from https://huggingface.co/laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K
+MODEL_PATH = ''
+_MODELS = {
+ "ViT-L/14": os.path.join(MODEL_PATH, "ViClip-InternVid-10M-FLT.pth"),
+}
+
+
+class QuickGELU(nn.Module):
+ def forward(self, x):
+ return x * torch.sigmoid(1.702 * x)
+
+
+class ResidualAttentionBlock(nn.Module):
+ def __init__(self, d_model, n_head, drop_path=0., attn_mask=None, dropout=0.):
+ super().__init__()
+
+ self.drop_path1 = DropPath(drop_path) if drop_path > 0. else nn.Identity()
+ self.drop_path2 = DropPath(drop_path) if drop_path > 0. else nn.Identity()
+ # logger.info(f'Droppath: {drop_path}')
+ self.attn = nn.MultiheadAttention(d_model, n_head, dropout=dropout)
+ self.ln_1 = nn.LayerNorm(d_model)
+ self.mlp = nn.Sequential(OrderedDict([
+ ("c_fc", nn.Linear(d_model, d_model * 4)),
+ ("gelu", QuickGELU()),
+ ("drop1", nn.Dropout(dropout)),
+ ("c_proj", nn.Linear(d_model * 4, d_model)),
+ ("drop2", nn.Dropout(dropout)),
+ ]))
+ self.ln_2 = nn.LayerNorm(d_model)
+ self.attn_mask = attn_mask
+
+ def attention(self, x):
+ self.attn_mask = self.attn_mask.to(dtype=x.dtype, device=x.device) if self.attn_mask is not None else None
+ return self.attn(x, x, x, need_weights=False, attn_mask=self.attn_mask)[0]
+
+ def forward(self, x):
+ x = x + self.drop_path1(self.attention(self.ln_1(x)))
+ x = x + self.drop_path2(self.mlp(self.ln_2(x)))
+ return x
+
+
+class Transformer(nn.Module):
+ def __init__(self, width, layers, heads, drop_path=0., checkpoint_num=0, dropout=0.):
+ super().__init__()
+ dpr = [x.item() for x in torch.linspace(0, drop_path, layers)]
+ self.resblocks = nn.ModuleList()
+ for idx in range(layers):
+ self.resblocks.append(ResidualAttentionBlock(width, heads, drop_path=dpr[idx], dropout=dropout))
+ self.checkpoint_num = checkpoint_num
+
+ def forward(self, x):
+ for idx, blk in enumerate(self.resblocks):
+ if idx < self.checkpoint_num:
+ x = checkpoint.checkpoint(blk, x)
+ else:
+ x = blk(x)
+ return x
+
+
+class VisionTransformer(nn.Module):
+ def __init__(
+ self, input_resolution, patch_size, width, layers, heads, output_dim=None,
+ kernel_size=1, num_frames=8, drop_path=0, checkpoint_num=0, dropout=0.,
+ temp_embed=True,
+ ):
+ super().__init__()
+ self.output_dim = output_dim
+ self.conv1 = nn.Conv3d(
+ 3, width,
+ (kernel_size, patch_size, patch_size),
+ (kernel_size, patch_size, patch_size),
+ (0, 0, 0), bias=False
+ )
+
+ scale = width ** -0.5
+ self.class_embedding = nn.Parameter(scale * torch.randn(width))
+ self.positional_embedding = nn.Parameter(scale * torch.randn((input_resolution // patch_size) ** 2 + 1, width))
+ self.ln_pre = nn.LayerNorm(width)
+ if temp_embed:
+ self.temporal_positional_embedding = nn.Parameter(torch.zeros(1, num_frames, width))
+
+ self.transformer = Transformer(
+ width, layers, heads, drop_path=drop_path, checkpoint_num=checkpoint_num,
+ dropout=dropout)
+
+ self.ln_post = nn.LayerNorm(width)
+ if output_dim is not None:
+ self.proj = nn.Parameter(torch.empty(width, output_dim))
+ else:
+ self.proj = None
+
+ self.dropout = nn.Dropout(dropout)
+
+ def get_num_layers(self):
+ return len(self.transformer.resblocks)
+
+ @torch.jit.ignore
+ def no_weight_decay(self):
+ return {'positional_embedding', 'class_embedding', 'temporal_positional_embedding'}
+
+ def mask_tokens(self, inputs, masking_prob=0.0):
+ B, L, _ = inputs.shape
+
+ # This is different from text as we are masking a fix number of tokens
+ Lm = int(masking_prob * L)
+ masked_indices = torch.zeros(B, L)
+ indices = torch.argsort(torch.rand_like(masked_indices), dim=-1)[:, :Lm]
+ batch_indices = (
+ torch.arange(masked_indices.shape[0]).unsqueeze(-1).expand_as(indices)
+ )
+ masked_indices[batch_indices, indices] = 1
+
+ masked_indices = masked_indices.bool()
+
+ return inputs[~masked_indices].reshape(B, -1, inputs.shape[-1])
+
+ def forward(self, x, masking_prob=0.0):
+ x = self.conv1(x) # shape = [*, width, grid, grid]
+ B, C, T, H, W = x.shape
+ x = x.permute(0, 2, 3, 4, 1).reshape(B * T, H * W, C)
+
+ x = torch.cat([self.class_embedding.to(x.dtype) + torch.zeros(x.shape[0], 1, x.shape[-1], dtype=x.dtype, device=x.device), x], dim=1) # shape = [*, grid ** 2 + 1, width]
+ x = x + self.positional_embedding.to(x.dtype)
+
+ # temporal pos
+ cls_tokens = x[:B, :1, :]
+ x = x[:, 1:]
+ x = rearrange(x, '(b t) n m -> (b n) t m', b=B, t=T)
+ if hasattr(self, 'temporal_positional_embedding'):
+ if x.size(1) == 1:
+ # This is a workaround for unused parameter issue
+ x = x + self.temporal_positional_embedding.mean(1)
+ else:
+ x = x + self.temporal_positional_embedding
+ x = rearrange(x, '(b n) t m -> b (n t) m', b=B, t=T)
+
+ if masking_prob > 0.0:
+ x = self.mask_tokens(x, masking_prob)
+
+ x = torch.cat((cls_tokens, x), dim=1)
+
+ x = self.ln_pre(x)
+
+ x = x.permute(1, 0, 2) #BND -> NBD
+ x = self.transformer(x)
+
+ x = self.ln_post(x)
+
+ if self.proj is not None:
+ x = self.dropout(x[0]) @ self.proj
+ else:
+ x = x.permute(1, 0, 2) #NBD -> BND
+
+ return x
+
+
+def inflate_weight(weight_2d, time_dim, center=True):
+ logger.info(f'Init center: {center}')
+ if center:
+ weight_3d = torch.zeros(*weight_2d.shape)
+ weight_3d = weight_3d.unsqueeze(2).repeat(1, 1, time_dim, 1, 1)
+ middle_idx = time_dim // 2
+ weight_3d[:, :, middle_idx, :, :] = weight_2d
+ else:
+ weight_3d = weight_2d.unsqueeze(2).repeat(1, 1, time_dim, 1, 1)
+ weight_3d = weight_3d / time_dim
+ return weight_3d
+
+
+def load_state_dict(model, state_dict, input_resolution=224, patch_size=16, center=True):
+ state_dict_3d = model.state_dict()
+ for k in state_dict.keys():
+ if k in state_dict_3d.keys() and state_dict[k].shape != state_dict_3d[k].shape:
+ if len(state_dict_3d[k].shape) <= 2:
+ logger.info(f'Ignore: {k}')
+ continue
+ logger.info(f'Inflate: {k}, {state_dict[k].shape} => {state_dict_3d[k].shape}')
+ time_dim = state_dict_3d[k].shape[2]
+ state_dict[k] = inflate_weight(state_dict[k], time_dim, center=center)
+
+ pos_embed_checkpoint = state_dict['positional_embedding']
+ embedding_size = pos_embed_checkpoint.shape[-1]
+ num_patches = (input_resolution // patch_size) ** 2
+ orig_size = int((pos_embed_checkpoint.shape[-2] - 1) ** 0.5)
+ new_size = int(num_patches ** 0.5)
+ if orig_size != new_size:
+ logger.info(f'Pos_emb from {orig_size} to {new_size}')
+ extra_tokens = pos_embed_checkpoint[:1]
+ pos_tokens = pos_embed_checkpoint[1:]
+ pos_tokens = pos_tokens.reshape(-1, orig_size, orig_size, embedding_size).permute(0, 3, 1, 2)
+ pos_tokens = torch.nn.functional.interpolate(
+ pos_tokens, size=(new_size, new_size), mode='bicubic', align_corners=False)
+ pos_tokens = pos_tokens.permute(0, 2, 3, 1).flatten(0, 2)
+ new_pos_embed = torch.cat((extra_tokens, pos_tokens), dim=0)
+ state_dict['positional_embedding'] = new_pos_embed
+
+ message = model.load_state_dict(state_dict, strict=False)
+ logger.info(f"Load pretrained weights: {message}")
+
+
+@register_model
+def clip_joint_b16(
+ pretrained=True, input_resolution=224, kernel_size=1,
+ center=True, num_frames=8, drop_path=0.
+):
+ model = VisionTransformer(
+ input_resolution=input_resolution, patch_size=16,
+ width=768, layers=12, heads=12, output_dim=512,
+ kernel_size=kernel_size, num_frames=num_frames,
+ drop_path=drop_path,
+ )
+ raise NotImplementedError
+ if pretrained:
+ logger.info('load pretrained weights')
+ state_dict = torch.load(_MODELS["ViT-B/16"], map_location='cpu')
+ load_state_dict(model, state_dict, input_resolution=input_resolution, patch_size=16, center=center)
+ return model.eval()
+
+
+@register_model
+def clip_joint_l14(
+ pretrained=False, input_resolution=224, kernel_size=1,
+ center=True, num_frames=8, drop_path=0., checkpoint_num=0,
+ dropout=0.,
+):
+ model = VisionTransformer(
+ input_resolution=input_resolution, patch_size=14,
+ width=1024, layers=24, heads=16, output_dim=768,
+ kernel_size=kernel_size, num_frames=num_frames,
+ drop_path=drop_path, checkpoint_num=checkpoint_num,
+ dropout=dropout,
+ )
+
+ if pretrained:
+ if isinstance(pretrained, str):
+ model_name = pretrained
+ else:
+ model_name = "ViT-L/14"
+ logger.info('load pretrained weights')
+ state_dict = torch.load(_MODELS[model_name], map_location='cpu')
+ load_state_dict(model, state_dict, input_resolution=input_resolution, patch_size=14, center=center)
+ return model.eval()
+
+
+@register_model
+def clip_joint_l14_336(
+ pretrained=True, input_resolution=336, kernel_size=1,
+ center=True, num_frames=8, drop_path=0.
+):
+ raise NotImplementedError
+ model = VisionTransformer(
+ input_resolution=input_resolution, patch_size=14,
+ width=1024, layers=24, heads=16, output_dim=768,
+ kernel_size=kernel_size, num_frames=num_frames,
+ drop_path=drop_path,
+ )
+ if pretrained:
+ logger.info('load pretrained weights')
+ state_dict = torch.load(_MODELS["ViT-L/14_336"], map_location='cpu')
+ load_state_dict(model, state_dict, input_resolution=input_resolution, patch_size=14, center=center)
+ return model.eval()
+
+
+def interpolate_pos_embed_vit(state_dict, new_model):
+ key = "vision_encoder.temporal_positional_embedding"
+ if key in state_dict:
+ vision_temp_embed_new = new_model.state_dict()[key]
+ vision_temp_embed_new = vision_temp_embed_new.unsqueeze(2) # [1, n, d] -> [1, n, 1, d]
+ vision_temp_embed_old = state_dict[key]
+ vision_temp_embed_old = vision_temp_embed_old.unsqueeze(2)
+
+ state_dict[key] = load_temp_embed_with_mismatch(
+ vision_temp_embed_old, vision_temp_embed_new, add_zero=False
+ ).squeeze(2)
+
+ key = "text_encoder.positional_embedding"
+ if key in state_dict:
+ text_temp_embed_new = new_model.state_dict()[key]
+ text_temp_embed_new = text_temp_embed_new.unsqueeze(0).unsqueeze(2) # [n, d] -> [1, n, 1, d]
+ text_temp_embed_old = state_dict[key]
+ text_temp_embed_old = text_temp_embed_old.unsqueeze(0).unsqueeze(2)
+
+ state_dict[key] = load_temp_embed_with_mismatch(
+ text_temp_embed_old, text_temp_embed_new, add_zero=False
+ ).squeeze(2).squeeze(0)
+ return state_dict
+
+
+if __name__ == '__main__':
+ import time
+ from fvcore.nn import FlopCountAnalysis
+ from fvcore.nn import flop_count_table
+ import numpy as np
+
+ seed = 4217
+ np.random.seed(seed)
+ torch.manual_seed(seed)
+ torch.cuda.manual_seed(seed)
+ torch.cuda.manual_seed_all(seed)
+ num_frames = 8
+
+ # model = clip_joint_b16(pretrained=True, kernel_size=1, num_frames=8, num_classes=400, drop_path=0.1)
+ # logger.info(model)
+ model = clip_joint_l14(pretrained=False)
+
+ flops = FlopCountAnalysis(model, torch.rand(1, 3, num_frames, 224, 224))
+ s = time.time()
+ logger.info(flop_count_table(flops, max_depth=1))
+ logger.info(time.time()-s)
+ # logger.info(model(torch.rand(1, 3, num_frames, 224, 224)).shape)
diff --git a/Data/instruction_data/README.md b/Data/instruction_data/README.md
new file mode 100644
index 0000000..9abf8f3
--- /dev/null
+++ b/Data/instruction_data/README.md
@@ -0,0 +1,41 @@
+# Instruction data for [VideoChat](https://github.com/OpenGVLab/Ask-Anything/tree/main/video_chat)
+
+# :fire: Updates
+- **2023/05/11**: Release the **V1**: [Google Drive](https://drive.google.com/file/d/1C-7xmf42QUEi4ApXTcxBHr5nLvTWXyUi/view?usp=sharing) | [Aliyun OSS](https://pjlab-gvm-data.oss-cn-shanghai.aliyuncs.com/data/videochat/videochat_instruct_11k.json)
+
+# :speech_balloon: V1: 7K detailed descriptions + 4K multi-turn conversations
+
+ We build a video-centric multimodal instruction data based on WebVid-10M. The corresponding detailed descriptions and multi-turn conversations generations are produced by ChatGPT based on video text (aided by [**VideoChat-Text**](https://github.com/OpenGVLab/Ask-Anything/tree/main/video_chat_with_ChatGPT)) with several prompts concerning **spatiotemporal features**. Compared with detailed video descriptions, video conversations are introduced to further improve data diversity by introducing **temporal and casual features** in the video instruction data.
+
+
+Example of detailed video description
+
+
+
+
+
+
+Example of video conversation
+
+
+
+
+
+# :page_facing_up: Citation
+
+If you find this project useful in your research, please consider cite:
+```BibTeX
+@article{2023videochat,
+ title={VideoChat: Chat-Centric Video Understanding},
+ author={Li, Kunchang and He, Yinan and Wang, Yi and Li, Yizhuo and Wang, Wenhai and Luo, Ping and Wang, Yali and Wang, Limin and Qiao, Yu},
+ journal={arXiv preprint arXiv:2305.06355},
+ year={2023}
+}
+
+@article{wang2022internvideo,
+ title={InternVideo: General Video Foundation Models via Generative and Discriminative Learning},
+ author={Wang, Yi and Li, Kunchang and Li, Yizhuo and He, Yinan and Huang, Bingkun and Zhao, Zhiyu and Zhang, Hongjie and Xu, Jilan and Liu, Yi and Wang, Zun and Xing, Sen and Chen, Guo and Pan, Junting and Yu, Jiashuo and Wang, Yali and Wang, Limin and Qiao, Yu},
+ journal={arXiv preprint arXiv:2212.03191},
+ year={2022}
+}
+```
diff --git a/Data/instruction_data/assert/conversation.png b/Data/instruction_data/assert/conversation.png
new file mode 100644
index 0000000..64220b0
Binary files /dev/null and b/Data/instruction_data/assert/conversation.png differ
diff --git a/Data/instruction_data/assert/detailed_description.png b/Data/instruction_data/assert/detailed_description.png
new file mode 100644
index 0000000..6a93076
Binary files /dev/null and b/Data/instruction_data/assert/detailed_description.png differ
diff --git a/Downstream/Open-Set-Action-Recognition/.gitignore b/Downstream/Open-Set-Action-Recognition/.gitignore
new file mode 100644
index 0000000..c8a1332
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/.gitignore
@@ -0,0 +1,143 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+**/*.pyc
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+# Usually these files are written by a python script from a template
+# before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+.github
+.pylintrc
+.pre-commit-config.yaml
+.readthedocs.yml
+
+# custom
+/data
+.vscode
+.idea
+*.pkl
+*.pkl.json
+*.log.json
+benchlist.txt
+work_dirs/
+
+# Pytorch
+*.pth
+
+# Profile
+*.prof
+
+# added by wentao
+*.avi
+temp/
+*.npz
+*.pkl
+experiments/*/results*
+experiments/*/logs
+*.o
+*backup*
+tools/data/mit/log*
+tools/data/*log*
+.github/workflows
+demo/hmdb51*
+demo/ucf101*
+demo/*.avi
+demo/*.mp4
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/README.md b/Downstream/Open-Set-Action-Recognition/README.md
new file mode 100644
index 0000000..cd6bd3d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/README.md
@@ -0,0 +1,112 @@
+# Open Set Action Recognition
+
+## Table of Contents
+1. [Introduction](#introduction)
+1. [Installation](#installation)
+1. [Datasets](#datasets)
+1. [Testing](#testing)
+1. [Training](#training)
+1. [Model Zoo](#model-zoo)
+
+
+## Introduction
+VideoIntern not only recognizes known classes accurately but also has a strong perception ability for unknown classes that are out of training classes. This repo is one of the generalization tasks——Open Set Action Recognition. Specifically, we finetune VideoMAE backbone with a linear classification head on UCF-101 dataset from the evidential deep learning (EDL) perspective, without any model calibration methods used in [DEAR](https://github.com/Cogito2012/DEAR). Our VideoIntern model achieves significant and consistent performance gains compared to multiple action recognition backbones (i.e., I3D, TSM, SlowFast, TPN), which are trained in the DEAR way, with HMDB-51 or MiT-v2 dataset as the unknown.
+
+## Installation
+This repo is developed from [MMAction2](https://github.com/open-mmlab/mmaction2) codebase.
+
+### Installation Steps
+a. Create a conda virtual environment of this repo, and activate it:
+
+```shell
+conda create -n OSAR python=3.7 -y
+conda activate OSAR
+```
+
+b. Install PyTorch and TorchVision following the [official instructions](https://pytorch.org/), e.g.,
+
+```shell
+conda install pytorch=1.7.0 cudatoolkit=11.0 torchvision=0.8.0 -c pytorch
+```
+c. Install mmcv, we recommend you to install the pre-build mmcv as below.
+
+```shell
+pip install mmcv-full==1.2.2 -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html
+```
+**Important:** If you have already installed `mmcv` and try to install `mmcv-full`, you have to uninstall `mmcv` first by running `pip uninstall mmcv`. Otherwise, there will be `ModuleNotFoundError`.
+
+d. Clone the source code of this repo:
+
+```shell
+git clone https://github.com/VideoIntern/Open-Set-Action-Recognition.git Open-Set-Action-Recognition-main
+cd Open-Set-Action-Recognition-main
+```
+e. Install build requirements and then install OSAR.
+
+```shell
+pip install -r requirements/build.txt
+python setup.py develop
+```
+
+If no error appears in your installation steps, then you are all set!
+
+## Datasets
+
+This repo uses standard video action datasets, i.e., UCF-101 for closed set training, and HMDB-51 and MiT-v2 test sets as two different unknowns. Please refer to the default [MMAction2 dataset setup steps](/tools/data/ucf101/README.md) to setup these three datasets correctly.
+
+**Note**: You can just ignore the `Step 3. Extract RGB and Flow` in the referred setup steps since all codes related to our paper do not rely on extracted frames and optical flow. This will save you large amount of disk space!
+
+## Testing
+
+To test our pre-trained models (see the [Model Zoo](#model-zoo)), you need to download a model file and unzip it under `work_dir`. Let's take the `I3D`-based DEAR model as an example. First, download the [pre-trained I3D-based models](https://drive.google.com/drive/folders/1TguABfmy0PE6jx9fflePQySe3jXWnsc0?usp=sharing), where the full DEAR model is saved in the folder `finetune_ucf101_i3d_edlnokl_avuc_debias`. The following directory tree is for your reference to place the downloaded files.
+```shell
+work_dirs
+├── mae
+│ ├── finetune_ucf101_mae_edlnokl
+│ │ └── latest.pth
+
+```
+a. Get Uncertainty Threshold.
+The threshold value of one model will be reported.
+```shell
+cd experiments/mae
+# run the thresholding with BATCH_SIZE=16 on 8 GPUs
+bash run_get_mae_threshold.sh edlnokl 16 8
+```
+
+b. Out-of-Distribution Detection.
+The uncertainty distribution figure of a specified model will be reported.
+```shell
+cd experiments/mae
+bash run_ood_mae_dist_detection.sh HMDB edlnokl 8
+```
+
+c. Compute AUROC.
+The AUROC score of a specified model will be reported.
+```shell
+cd experiments/mae/results
+python compute_auroc.py
+```
+
+## Training
+
+```shell
+cd experiments/mae
+bash finetune_mae_edlnokl_ucf101.sh 8
+```
+
+## Model Zoo
+
+The pre-trained weights (checkpoints) are available below.
+| Model | Checkpoint | Train Config | Test Config | Open Set AUC (%) | Closed Set ACC (%) |
+|:--|:--:|:--:|:--:|:--:|:--:|
+|I3D + DEAR |[ckpt](https://drive.google.com/file/d/1oRNBH0aAhFpcJSBqWlT4x0ru7iHfdndW/view?usp=sharing)| [train](configs/recognition/i3d/finetune_ucf101_i3d_edlnokl_avuc_debias.py) | [test](configs/recognition/i3d/inference_i3d_enn.py) | 77.08 / 81.54 | 93.89 |
+|TSM + DEAR | [ckpt](https://drive.google.com/file/d/1TM1c28jRyZpOrWqwaQPYXFBZJXHQp__9/view?usp=sharing)| [train](configs/recognition/tsm/finetune_ucf101_tsm_edlnokl_avuc_debias.py) | [test](configs/recognition/tsm/inference_tsm_enn.py) | 78.65 / 83.92 | 94.48 |
+|TPN + DEAR | [ckpt](https://drive.google.com/file/d/1jorfFMMzWd5xDCfZsemoWD8Rg7DbH16u/view?usp=sharing)| [train](configs/recognition/tpn/tpn_slowonly_edlloss_nokl_avuc_debias_r50_8x8x1_150e_kinetics_rgb.py) | [test](configs/recognition/tpn/inference_tpn_slowonly_enn.py) | 79.23 / 81.80 | 96.30 |
+|SlowFast + DEAR |[ckpt](https://drive.google.com/file/d/13LNRv0BYkVfzCA95RB5dCp53MmErRL5D/view?usp=sharing)| [train](configs/recognition/slowfast/finetune_ucf101_slowfast_edlnokl_avuc_debias.py) | [test](configs/recognition/slowfast/inference_slowfast_enn.py) | 82.94 / 86.99 | 96.48 |
+|InternVideo-B + EDL |[ckpt](https://drive.google.com/file/d/1lW1mHCbyfi0tvIxAjVzjr3g-AgK61ND3/view?usp=share_link)| [train](configs/recognition/mae/finetune_ucf101_mae_edlnokl.py) | [test](configs/recognition/mae/inference_mae_enn.py) | 83.21 / 88.98 | 96.91 |
+|InternVideo-L + EDL |[ckpt](https://drive.google.com/file/d/1lW1mHCbyfi0tvIxAjVzjr3g-AgK61ND3/view?usp=share_link)| [train](configs/recognition/mae/finetune_ucf101_mae_edlnokl.py) | [test](configs/recognition/mae/inference_mae_enn.py) | 83.82 / 91.13 | 97.36 |
+|InternVideo-H + EDL |[ckpt](https://drive.google.com/file/d/1lW1mHCbyfi0tvIxAjVzjr3g-AgK61ND3/view?usp=share_link)| [train](configs/recognition/mae/finetune_ucf101_mae_edlnokl.py) | [test](configs/recognition/mae/inference_mae_enn.py) | 85.48 / 91.85 | 97.89 |
+
+For the pretrained MAE model, please download it in the [Google Drive](https://drive.google.com/file/d/1iVb7c3onYPjIv5ResMRbIoxCVlsp5YCr/view?usp=share_link).
+
diff --git a/Downstream/Open-Set-Action-Recognition/configs/localization/bmn/README.md b/Downstream/Open-Set-Action-Recognition/configs/localization/bmn/README.md
new file mode 100644
index 0000000..22f0fcc
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/localization/bmn/README.md
@@ -0,0 +1,79 @@
+# BMN
+
+## Introduction
+```
+@inproceedings{lin2019bmn,
+ title={Bmn: Boundary-matching network for temporal action proposal generation},
+ author={Lin, Tianwei and Liu, Xiao and Li, Xin and Ding, Errui and Wen, Shilei},
+ booktitle={Proceedings of the IEEE International Conference on Computer Vision},
+ pages={3889--3898},
+ year={2019}
+}
+
+@article{zhao2017cuhk,
+ title={Cuhk \& ethz \& siat submission to activitynet challenge 2017},
+ author={Zhao, Y and Zhang, B and Wu, Z and Yang, S and Zhou, L and Yan, S and Wang, L and Xiong, Y and Lin, D and Qiao, Y and others},
+ journal={arXiv preprint arXiv:1710.08011},
+ volume={8},
+ year={2017}
+}
+```
+
+## Model Zoo
+
+### ActivityNet feature
+
+|config |feature | gpus | AR@100| AUC | AP@0.5 | AP@0.75 | AP@0.95 | mAP | gpu_mem(M) | iter time(s) | ckpt | log| json|
+|:-:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:-:|---|:-:|:-:|---|
+|[bmn_400x100_9e_2x8_activitynet_feature](/configs/localization/bmn/bmn_400x100_2x8_9e_activitynet_feature.py) |cuhk_mean_100 |2|75.28|67.22|42.47|31.31|9.92|30.34|5420|3.27|[ckpt](https://download.openmmlab.com/mmaction/localization/bmn/bmn_400x100_9e_activitynet_feature/bmn_400x100_9e_activitynet_feature_20200619-42a3b111.pth)| [log](https://download.openmmlab.com/mmaction/localization/bmn/bmn_400x100_9e_activitynet_feature/bmn_400x100_9e_activitynet_feature.log)| [json](https://download.openmmlab.com/mmaction/localization/bmn/bmn_400x100_9e_activitynet_feature/bmn_400x100_9e_activitynet_feature.log.json)|
+| |mmaction_video |2|75.43|67.22|42.62|31.56|10.86|30.77|5420|3.27|[ckpt](https://download.openmmlab.com/mmaction/localization/bmn/bmn_400x100_2x8_9e_mmaction_video/bmn_400x100_2x8_9e_mmaction_video_20200809-c9fd14d2.pth)| [log](https://download.openmmlab.com/mmaction/localization/bmn/bmn_400x100_2x8_9e_mmaction_video/bmn_400x100_2x8_9e_mmaction_video_20200809.log) | [json](https://download.openmmlab.com/mmaction/localization/bmn/bmn_400x100_2x8_9e_mmaction_video/bmn_400x100_2x8_9e_mmaction_video_20200809.json) |
+| |mmaction_clip |2|75.35|67.38|43.08|32.19|10.73|31.15|5420|3.27|[ckpt](https://download.openmmlab.com/mmaction/localization/bmn/bmn_400x100_2x8_9e_mmaction_clip/bmn_400x100_2x8_9e_mmaction_clip_20200809-10d803ce.pth)| [log](https://download.openmmlab.com/mmaction/localization/bmn/bmn_400x100_2x8_9e_mmaction_clip/bmn_400x100_2x8_9e_mmaction_clip_20200809.log) | [json](https://download.openmmlab.com/mmaction/localization/bmn/bmn_400x100_2x8_9e_mmaction_clip/bmn_400x100_2x8_9e_mmaction_clip_20200809.json) |
+| [BMN-official](https://github.com/JJBOY/BMN-Boundary-Matching-Network) (for reference)* |cuhk_mean_100 |-|75.27|67.49|42.22|30.98|9.22|30.00|-|-|-| - | - |
+
+- Notes:
+
+1. The **gpus** indicates the number of gpu we used to get the checkpoint.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. For feature column, cuhk_mean_100 denotes the widely used cuhk activitynet feature extracted by [anet2016-cuhk](https://github.com/yjxiong/anet2016-cuhk), mmaction_video and mmaction_clip denote feature extracted by mmaction, with video-level activitynet finetuned model or clip-level activitynet finetuned model respectively.
+3. We evaluate the action detection performance of BMN, using [anet_cuhk_2017](https://download.openmmlab.com/mmaction/localization/cuhk_anet17_pred.json) submission for ActivityNet2017 Untrimmed Video Classification Track to assign label for each action proposal.
+
+*We train BMN with the [official repo](https://github.com/JJBOY/BMN-Boundary-Matching-Network), evaluate its proposal generation and action detection performance with [anet_cuhk_2017](https://download.openmmlab.com/mmaction/localization/cuhk_anet17_pred.json) for label assigning.
+
+For more details on data preparation, you can refer to ActivityNet feature in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train BMN model on ActivityNet features dataset.
+```shell
+python tools/train.py configs/localization/bmn/bmn_400x100_2x8_9e_activitynet_feature.py
+```
+For more details and optional arguments infos, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting) .
+
+## Test
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test BMN on ActivityNet feature dataset.
+```shell
+# Note: If evaluated, then please make sure the annotation file for test data contains groundtruth.
+python tools/test.py configs/localization/bmn/bmn_400x100_2x8_9e_activitynet_feature.py checkpoints/SOME_CHECKPOINT.pth --eval AR@AN --out results.json
+```
+You can also test the action detection performance of the model, with [anet_cuhk_2017](https://download.openmmlab.com/mmaction/localization/cuhk_anet17_pred.json) prediction file and generated proposal file (`results.json` in last command).
+```shell
+python tools/analysis/report_map.py --proposal path/to/proposal_file
+```
+
+Notes:
+1. (Optional) You can use the following command to generate a formatted proposal file, which will be fed into the action classifier (Currently supports SSN and P-GCN, not including TSN, I3D etc.) to get the classification result of proposals.
+ ```shell
+ python tools/data/activitynet/convert_proposal_format.py
+ ```
+
+For more details and optional arguments infos, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset) .
diff --git a/Downstream/Open-Set-Action-Recognition/configs/localization/bmn/bmn_400x100_2x8_9e_activitynet_feature.py b/Downstream/Open-Set-Action-Recognition/configs/localization/bmn/bmn_400x100_2x8_9e_activitynet_feature.py
new file mode 100644
index 0000000..753bc73
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/localization/bmn/bmn_400x100_2x8_9e_activitynet_feature.py
@@ -0,0 +1,105 @@
+# model settings
+model = dict(
+ type='BMN',
+ temporal_dim=100,
+ boundary_ratio=0.5,
+ num_samples=32,
+ num_samples_per_bin=3,
+ feat_dim=400,
+ soft_nms_alpha=0.4,
+ soft_nms_low_threshold=0.5,
+ soft_nms_high_threshold=0.9,
+ post_process_top_k=100)
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='score')
+# dataset settings
+dataset_type = 'ActivityNetDataset'
+data_root = 'data/ActivityNet/activitynet_feature_cuhk/csv_mean_100/'
+data_root_val = 'data/ActivityNet/activitynet_feature_cuhk/csv_mean_100/'
+ann_file_train = 'data/ActivityNet/anet_anno_train.json'
+ann_file_val = 'data/ActivityNet/anet_anno_val.json'
+ann_file_test = 'data/ActivityNet/anet_anno_val.json'
+
+test_pipeline = [
+ dict(type='LoadLocalizationFeature'),
+ dict(
+ type='Collect',
+ keys=['raw_feature'],
+ meta_name='video_meta',
+ meta_keys=[
+ 'video_name', 'duration_second', 'duration_frame', 'annotations',
+ 'feature_frame'
+ ]),
+ dict(type='ToTensor', keys=['raw_feature']),
+]
+train_pipeline = [
+ dict(type='LoadLocalizationFeature'),
+ dict(type='GenerateLocalizationLabels'),
+ dict(
+ type='Collect',
+ keys=['raw_feature', 'gt_bbox'],
+ meta_name='video_meta',
+ meta_keys=['video_name']),
+ dict(type='ToTensor', keys=['raw_feature', 'gt_bbox']),
+ dict(
+ type='ToDataContainer',
+ fields=[dict(key='gt_bbox', stack=False, cpu_only=True)])
+]
+val_pipeline = [
+ dict(type='LoadLocalizationFeature'),
+ dict(type='GenerateLocalizationLabels'),
+ dict(
+ type='Collect',
+ keys=['raw_feature', 'gt_bbox'],
+ meta_name='video_meta',
+ meta_keys=[
+ 'video_name', 'duration_second', 'duration_frame', 'annotations',
+ 'feature_frame'
+ ]),
+ dict(type='ToTensor', keys=['raw_feature', 'gt_bbox']),
+ dict(
+ type='ToDataContainer',
+ fields=[dict(key='gt_bbox', stack=False, cpu_only=True)])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=8,
+ train_dataloader=dict(drop_last=True),
+ val_dataloader=dict(videos_per_gpu=1),
+ test_dataloader=dict(videos_per_gpu=1),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ pipeline=test_pipeline,
+ data_prefix=data_root_val),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ pipeline=val_pipeline,
+ data_prefix=data_root_val),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ pipeline=train_pipeline,
+ data_prefix=data_root))
+
+# optimizer
+optimizer = dict(
+ type='Adam', lr=0.001, weight_decay=0.0001) # this lr is used for 2 gpus
+optimizer_config = dict(grad_clip=None)
+# learning policy
+lr_config = dict(policy='step', step=7)
+
+total_epochs = 9
+checkpoint_config = dict(interval=1)
+evaluation = dict(interval=1, metrics=['AR@AN'])
+log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/bmn_400x100_2x8_9e_activitynet_feature/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+output_config = dict(out=f'{work_dir}/results.json', output_format='json')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/README.md b/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/README.md
new file mode 100644
index 0000000..69ce33e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/README.md
@@ -0,0 +1,131 @@
+# BSN
+
+## Introduction
+```
+@inproceedings{lin2018bsn,
+ title={Bsn: Boundary sensitive network for temporal action proposal generation},
+ author={Lin, Tianwei and Zhao, Xu and Su, Haisheng and Wang, Chongjing and Yang, Ming},
+ booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
+ pages={3--19},
+ year={2018}
+}
+```
+
+## Model Zoo
+
+### ActivityNet feature
+
+|config |feature | gpus| pretrain | AR@100| AUC | gpu_mem(M) | iter time(s) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:-:|
+|bsn_400x100_1x16_20e_activitynet_feature |cuhk_mean_100 |1| None |74.65|66.45|41(TEM)+25(PEM)|0.074(TEM)+0.036(PEM)|[ckpt_tem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_tem_400x100_1x16_20e_activitynet_feature/bsn_tem_400x100_1x16_20e_activitynet_feature_20200619-cd6accc3.pth) [ckpt_pem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_pem_400x100_1x16_20e_activitynet_feature/bsn_pem_400x100_1x16_20e_activitynet_feature_20200619-6111891d.pth)| [log_tem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_tem_400x100_1x16_20e_activitynet_feature/bsn_tem_400x100_1x16_20e_activitynet_feature.log) [log_pem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_pem_400x100_1x16_20e_activitynet_feature/bsn_pem_400x100_1x16_20e_activitynet_feature.log)| [json_tem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_tem_400x100_1x16_20e_activitynet_feature/bsn_tem_400x100_1x16_20e_activitynet_feature.log.json) [json_pem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_pem_400x100_1x16_20e_activitynet_feature/bsn_pem_400x100_1x16_20e_activitynet_feature.log.json)|
+| |mmaction_video |1| None |74.93|66.74|41(TEM)+25(PEM)|0.074(TEM)+0.036(PEM)|[ckpt_tem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_tem_400x100_1x16_20e_mmaction_video/bsn_tem_400x100_1x16_20e_mmaction_video_20200809-ad6ec626.pth) [ckpt_pem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_pem_400x100_1x16_20e_mmaction_video/bsn_pem_400x100_1x16_20e_mmaction_video_20200809-aa861b26.pth)| [log_tem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_tem_400x100_1x16_20e_mmaction_video/bsn_tem_400x100_1x16_20e_mmaction_video_20200809.log) [log_pem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_pem_400x100_1x16_20e_mmaction_video/bsn_pem_400x100_1x16_20e_mmaction_video_20200809.log) | [json_tem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_tem_400x100_1x16_20e_mmaction_video/bsn_tem_400x100_1x16_20e_mmaction_video_20200809.json) [json_pem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_pem_400x100_1x16_20e_mmaction_video/bsn_pem_400x100_1x16_20e_mmaction_video_20200809.json) |
+| |mmaction_clip |1| None |75.19|66.81|41(TEM)+25(PEM)|0.074(TEM)+0.036(PEM)|[ckpt_tem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_tem_400x100_1x16_20e_mmaction_clip/bsn_tem_400x100_1x16_20e_mmaction_clip_20200809-0a563554.pth) [ckpt_pem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_pem_400x100_1x16_20e_mmaction_clip/bsn_pem_400x100_1x16_20e_mmaction_clip_20200809-e32f61e6.pth)| [log_tem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_tem_400x100_1x16_20e_mmaction_clip/bsn_tem_400x100_1x16_20e_mmaction_clip_20200809.log) [log_pem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_pem_400x100_1x16_20e_mmaction_clip/bsn_pem_400x100_1x16_20e_mmaction_clip_20200809.log) | [json_tem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_tem_400x100_1x16_20e_mmaction_clip/bsn_tem_400x100_1x16_20e_mmaction_clip_20200809.json) [json_pem](https://download.openmmlab.com/mmaction/localization/bsn/bsn_pem_400x100_1x16_20e_mmaction_clip/bsn_pem_400x100_1x16_20e_mmaction_clip_20200809.json) |
+
+Notes:
+1. The **gpus** indicates the number of gpu we used to get the checkpoint.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. For feature column, cuhk_mean_100 denotes the widely used cuhk activitynet feature extracted by [anet2016-cuhk](https://github.com/yjxiong/anet2016-cuhk), mmaction_video and mmaction_clip denote feature extracted by mmaction, with video-level activitynet finetuned model or clip-level activitynet finetuned model respectively.
+
+For more details on data preparation, you can refer to ActivityNet feature in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+You can use the following commands to train a model.
+
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Examples:
+
+1. train BSN(TEM) on ActivityNet features dataset.
+ ```shell
+ python tools/train.py configs/localization/bsn/bsn_tem_400x100_1x16_20e_activitynet_feature.py
+ ```
+
+2. train BSN(PEM) on PGM results.
+ ```shell
+ python tools/train.py configs/localization/bsn/bsn_pem_400x100_1x16_20e_activitynet_feature.py
+ ```
+
+For more details and optional arguments infos, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Inference
+You can use the following commands to inference a model.
+
+1. For TEM Inference
+ ```shell
+ # Note: This could not be evaluated.
+ python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+ ```
+
+2. For PGM Inference
+ ```shell
+ python tools/bsn_proposal_generation.py ${CONFIG_FILE} [--mode ${MODE}]
+ ```
+
+3. For PEM Inference
+ ```shell
+ python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+ ```
+
+Examples:
+
+1. Inference BSN(TEM) with pretrained model.
+ ```shell
+ python tools/test.py configs/localization/bsn/bsn_tem_400x100_1x16_20e_activitynet_feature.py checkpoints/SOME_CHECKPOINT.pth
+ ```
+
+2. Inference BSN(PGM) with pretrained model.
+ ```shell
+ python tools/bsn_proposal_generation.py configs/localization/bsn/bsn_pgm_400x100_activitynet_feature.py --mode train
+ ```
+
+3. Inference BSN(PEM) with evaluation metric 'AR@AN' and output the results.
+ ```shell
+ # Note: If evaluated, then please make sure the annotation file for test data contains groundtruth.
+ python tools/test.py configs/localization/bsn/bsn_pem_400x100_1x16_20e_activitynet_feature.py checkpoints/SOME_CHECKPOINT.pth --eval AR@AN --out results.json
+ ```
+
+## Test
+You can use the following commands to test a model.
+
+1. TEM
+ ```shell
+ # Note: This could not be evaluated.
+ python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+ ```
+
+2. PGM
+ ```shell
+ python tools/bsn_proposal_generation.py ${CONFIG_FILE} [--mode ${MODE}]
+ ```
+
+3. PEM
+ ```shell
+ python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+ ```
+
+Examples:
+
+1. Test a TEM model on ActivityNet dataset.
+ ```shell
+ python tools/test.py configs/localization/bsn/bsn_tem_400x100_1x16_20e_activitynet_feature.py checkpoints/SOME_CHECKPOINT.pth
+ ```
+
+2. Test a PGM model on ActivityNet dataset.
+ ```shell
+ python tools/bsn_proposal_generation.py configs/localization/bsn/bsn_pgm_400x100_activitynet_feature.py --mode test
+ ```
+
+3. Test a PEM model with with evaluation metric 'AR@AN' and output the results.
+ ```shell
+ python tools/test.py configs/localization/bsn/bsn_pem_400x100_1x16_20e_activitynet_feature.py checkpoints/SOME_CHECKPOINT.pth --eval AR@AN --out results.json
+ ```
+Notes:
+1. (Optional) You can use the following command to generate a formatted proposal file, which will be fed into the action classifier (Currently supports only SSN and P-GCN, not including TSN, I3D etc.) to get the classification result of proposals.
+ ```shell
+ python tools/data/activitynet/convert_proposal_format.py
+ ```
+
+For more details and optional arguments infos, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/bsn_pem_400x100_1x16_20e_activitynet_feature.py b/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/bsn_pem_400x100_1x16_20e_activitynet_feature.py
new file mode 100644
index 0000000..aff4f45
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/bsn_pem_400x100_1x16_20e_activitynet_feature.py
@@ -0,0 +1,122 @@
+# model settings
+model = dict(
+ type='PEM',
+ pem_feat_dim=32,
+ pem_hidden_dim=256,
+ pem_u_ratio_m=1,
+ pem_u_ratio_l=2,
+ pem_high_temporal_iou_threshold=0.6,
+ pem_low_temporal_iou_threshold=2.2,
+ soft_nms_alpha=0.75,
+ soft_nms_low_threshold=0.65,
+ soft_nms_high_threshold=0.9,
+ post_process_top_k=100)
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='score')
+# dataset settings
+dataset_type = 'ActivityNetDataset'
+data_root = 'data/ActivityNet/activitynet_feature_cuhk/csv_mean_100/'
+data_root_val = 'data/ActivityNet/activitynet_feature_cuhk/csv_mean_100/'
+ann_file_train = 'data/ActivityNet/anet_anno_train.json'
+ann_file_val = 'data/ActivityNet/anet_anno_val.json'
+ann_file_test = 'data/ActivityNet/anet_anno_val.json'
+
+work_dir = 'work_dirs/bsn_400x100_20e_1x16_activitynet_feature/'
+pgm_proposals_dir = f'{work_dir}/pgm_proposals/'
+pgm_features_dir = f'{work_dir}/pgm_features/'
+
+test_pipeline = [
+ dict(
+ type='LoadProposals',
+ top_k=1000,
+ pgm_proposals_dir=pgm_proposals_dir,
+ pgm_features_dir=pgm_features_dir),
+ dict(
+ type='Collect',
+ keys=['bsp_feature', 'tmin', 'tmax', 'tmin_score', 'tmax_score'],
+ meta_name='video_meta',
+ meta_keys=[
+ 'video_name', 'duration_second', 'duration_frame', 'annotations',
+ 'feature_frame'
+ ]),
+ dict(type='ToTensor', keys=['bsp_feature'])
+]
+
+train_pipeline = [
+ dict(
+ type='LoadProposals',
+ top_k=500,
+ pgm_proposals_dir=pgm_proposals_dir,
+ pgm_features_dir=pgm_features_dir),
+ dict(
+ type='Collect',
+ keys=['bsp_feature', 'reference_temporal_iou'],
+ meta_name='video_meta',
+ meta_keys=[]),
+ dict(type='ToTensor', keys=['bsp_feature', 'reference_temporal_iou']),
+ dict(
+ type='ToDataContainer',
+ fields=(dict(key='bsp_feature', stack=False),
+ dict(key='reference_temporal_iou', stack=False)))
+]
+
+val_pipeline = [
+ dict(
+ type='LoadProposals',
+ top_k=1000,
+ pgm_proposals_dir=pgm_proposals_dir,
+ pgm_features_dir=pgm_features_dir),
+ dict(
+ type='Collect',
+ keys=['bsp_feature', 'tmin', 'tmax', 'tmin_score', 'tmax_score'],
+ meta_name='video_meta',
+ meta_keys=[
+ 'video_name', 'duration_second', 'duration_frame', 'annotations',
+ 'feature_frame'
+ ]),
+ dict(type='ToTensor', keys=['bsp_feature'])
+]
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=8,
+ train_dataloader=dict(drop_last=True),
+ val_dataloader=dict(videos_per_gpu=1),
+ test_dataloader=dict(videos_per_gpu=1),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ pipeline=test_pipeline,
+ data_prefix=data_root_val),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ pipeline=val_pipeline,
+ data_prefix=data_root_val),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ pipeline=train_pipeline,
+ data_prefix=data_root))
+
+# optimizer
+optimizer = dict(
+ type='Adam', lr=0.01, weight_decay=0.00001) # this lr is used for 1 gpus
+
+optimizer_config = dict(grad_clip=None)
+# learning policy
+lr_config = dict(policy='step', step=10)
+
+total_epochs = 20
+checkpoint_config = dict(interval=1, filename_tmpl='pem_epoch_{}.pth')
+
+evaluation = dict(interval=1, metrics=['AR@AN'])
+
+log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+output_config = dict(out=f'{work_dir}/results.json', output_format='json')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/bsn_pgm_400x100_activitynet_feature.py b/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/bsn_pgm_400x100_activitynet_feature.py
new file mode 100644
index 0000000..2c5f7a0
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/bsn_pgm_400x100_activitynet_feature.py
@@ -0,0 +1,32 @@
+# dataset settings
+dataset_type = 'ActivityNetDataset'
+data_root = 'data/ActivityNet/activitynet_feature_cuhk/csv_mean_100/'
+data_root_val = 'data/ActivityNet/activitynet_feature_cuhk/csv_mean_100/'
+ann_file_train = 'data/ActivityNet/anet_anno_train.json'
+ann_file_val = 'data/ActivityNet/anet_anno_val.json'
+ann_file_test = 'data/ActivityNet/anet_anno_test.json'
+
+work_dir = 'work_dirs/bsn_400x100_20e_1x16_activitynet_feature/'
+tem_results_dir = f'{work_dir}/tem_results/'
+pgm_proposals_dir = f'{work_dir}/pgm_proposals/'
+pgm_features_dir = f'{work_dir}/pgm_features/'
+
+temporal_scale = 100
+pgm_proposals_cfg = dict(
+ pgm_proposals_thread=8, temporal_scale=temporal_scale, peak_threshold=0.5)
+pgm_features_test_cfg = dict(
+ pgm_features_thread=4,
+ top_k=1000,
+ num_sample_start=8,
+ num_sample_end=8,
+ num_sample_action=16,
+ num_sample_interp=3,
+ bsp_boundary_ratio=0.2)
+pgm_features_train_cfg = dict(
+ pgm_features_thread=4,
+ top_k=500,
+ num_sample_start=8,
+ num_sample_end=8,
+ num_sample_action=16,
+ num_sample_interp=3,
+ bsp_boundary_ratio=0.2)
diff --git a/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/bsn_tem_400x100_1x16_20e_activitynet_feature.py b/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/bsn_tem_400x100_1x16_20e_activitynet_feature.py
new file mode 100644
index 0000000..bcb1a18
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/localization/bsn/bsn_tem_400x100_1x16_20e_activitynet_feature.py
@@ -0,0 +1,95 @@
+# model settings
+model = dict(
+ type='TEM',
+ temporal_dim=100,
+ boundary_ratio=0.1,
+ tem_feat_dim=400,
+ tem_hidden_dim=512,
+ tem_match_threshold=0.5)
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='score')
+# dataset settings
+dataset_type = 'ActivityNetDataset'
+data_root = 'data/ActivityNet/activitynet_feature_cuhk/csv_mean_100/'
+data_root_val = 'data/ActivityNet/activitynet_feature_cuhk/csv_mean_100/'
+ann_file_train = 'data/ActivityNet/anet_anno_train.json'
+ann_file_val = 'data/ActivityNet/anet_anno_val.json'
+ann_file_test = 'data/ActivityNet/anet_anno_full.json'
+
+work_dir = 'work_dirs/bsn_400x100_20e_1x16_activitynet_feature/'
+tem_results_dir = f'{work_dir}/tem_results/'
+
+test_pipeline = [
+ dict(type='LoadLocalizationFeature'),
+ dict(
+ type='Collect',
+ keys=['raw_feature'],
+ meta_name='video_meta',
+ meta_keys=['video_name']),
+ dict(type='ToTensor', keys=['raw_feature'])
+]
+train_pipeline = [
+ dict(type='LoadLocalizationFeature'),
+ dict(type='GenerateLocalizationLabels'),
+ dict(
+ type='Collect',
+ keys=['raw_feature', 'gt_bbox'],
+ meta_name='video_meta',
+ meta_keys=['video_name']),
+ dict(type='ToTensor', keys=['raw_feature', 'gt_bbox']),
+ dict(type='ToDataContainer', fields=[dict(key='gt_bbox', stack=False)])
+]
+val_pipeline = [
+ dict(type='LoadLocalizationFeature'),
+ dict(type='GenerateLocalizationLabels'),
+ dict(
+ type='Collect',
+ keys=['raw_feature', 'gt_bbox'],
+ meta_name='video_meta',
+ meta_keys=['video_name']),
+ dict(type='ToTensor', keys=['raw_feature', 'gt_bbox']),
+ dict(type='ToDataContainer', fields=[dict(key='gt_bbox', stack=False)])
+]
+
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=8,
+ train_dataloader=dict(drop_last=True),
+ val_dataloader=dict(videos_per_gpu=1),
+ test_dataloader=dict(videos_per_gpu=1),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ pipeline=test_pipeline,
+ data_prefix=data_root_val),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ pipeline=val_pipeline,
+ data_prefix=data_root_val),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ pipeline=train_pipeline,
+ data_prefix=data_root))
+
+# optimizer
+optimizer = dict(
+ type='Adam', lr=0.001, weight_decay=0.0001) # this lr is used for 1 gpus
+
+optimizer_config = dict(grad_clip=None)
+# learning policy
+lr_config = dict(policy='step', step=7)
+
+total_epochs = 20
+checkpoint_config = dict(interval=1, filename_tmpl='tem_epoch_{}.pth')
+
+log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1), ('val', 1)]
+output_config = dict(out=tem_results_dir, output_format='csv')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/localization/ssn/README.md b/Downstream/Open-Set-Action-Recognition/configs/localization/ssn/README.md
new file mode 100644
index 0000000..c834dbd
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/localization/ssn/README.md
@@ -0,0 +1,55 @@
+# SSN
+
+## Introduction
+```
+@InProceedings{Zhao_2017_ICCV,
+author = {Zhao, Yue and Xiong, Yuanjun and Wang, Limin and Wu, Zhirong and Tang, Xiaoou and Lin, Dahua},
+title = {Temporal Action Detection With Structured Segment Networks},
+booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
+month = {Oct},
+year = {2017}
+}
+```
+
+## Model Zoo
+
+| config | gpus | backbone | pretrain | mAP@0.3 | mAP@0.4 | mAP@0.5 | reference mAP@0.3 | reference mAP@0.4 | reference mAP@0.5 | gpu_mem(M) | ckpt | log | json | refrence ckpt | refrence json
+|:-:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:-:|:-:|:-:|:-:|---|:--:|:--:|
+|[ssn_r50_450e_thumos14_rgb](/configs/localization/ssn/ssn_r50_450e_thumos14_rgb_train.py) |8| ResNet50 | ImageNet |29.37|22.15|15.69|[27.61](https://github.com/open-mmlab/mmaction/tree/c7e3b7c11fb94131be9b48a8e3d510589addc3ce#Get%20started)|[21.28](https://github.com/open-mmlab/mmaction/tree/c7e3b7c11fb94131be9b48a8e3d510589addc3ce#Get%20started)|[14.57](https://github.com/open-mmlab/mmaction/tree/c7e3b7c11fb94131be9b48a8e3d510589addc3ce#Get%20started)|6352|[ckpt](https://download.openmmlab.com/mmaction/localization/ssn/ssn_r50_450e_thumos14_rgb/ssn_r50_450e_thumos14_rgb_20201012-1920ab16.pth)| [log](https://download.openmmlab.com/mmaction/localization/ssn/ssn_r50_450e_thumos14_rgb/20201005_144656.log)| [json](https://download.openmmlab.com/mmaction/localization/ssn/ssn_r50_450e_thumos14_rgb/20201005_144656.log.json)| [ckpt](https://download.openmmlab.com/mmaction/localization/ssn/mmaction_reference/ssn_r50_450e_thumos14_rgb_ref/ssn_r50_450e_thumos14_rgb_ref_20201014-b6f48f68.pth)| [json](https://download.openmmlab.com/mmaction/localization/ssn/mmaction_reference/ssn_r50_450e_thumos14_rgb_ref/20201008_103258.log.json)|
+
+- Notes:
+
+1. The **gpus** indicates the number of gpu we used to get the checkpoint.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. Since SSN utilizes different structured temporal pyramid pooling methods at training and testing, please refer to [ssn_r50_450e_thumos14_rgb_train](/configs/localization/ssn/ssn_r50_450e_thumos14_rgb_train.py) at training and [ssn_r50_450e_thumos14_rgb_test](/configs/localization/ssn/ssn_r50_450e_thumos14_rgb_test.py) at testing.
+3. We evaluate the action detection performance of SSN, using action proposals of TAG. For more details on data preparation, you can refer to thumos14 TAG proposals in [Data Preparation](/docs/data_preparation.md).
+4. The reference SSN in is evaluated with `ResNet50` backbone in MMAction, which is the same backbone with ours. Note that the original setting of MMAction SSN uses the `BNInception` backbone.
+
+## Train
+
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train SSN model on thumos14 dataset.
+```shell
+python tools/train.py configs/localization/ssn/ssn_r50_450e_thumos14_rgb_train.py
+```
+For more details and optional arguments infos, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting) .
+
+## Test
+
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test BMN on ActivityNet feature dataset.
+```shell
+# Note: If evaluated, then please make sure the annotation file for test data contains groundtruth.
+python tools/test.py configs/localization/ssn/ssn_r50_450e_thumos14_rgb_test.py checkpoints/SOME_CHECKPOINT.pth --eval mAP
+```
+
+For more details and optional arguments infos, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset) .
diff --git a/Downstream/Open-Set-Action-Recognition/configs/localization/ssn/ssn_r50_450e_thumos14_rgb_test.py b/Downstream/Open-Set-Action-Recognition/configs/localization/ssn/ssn_r50_450e_thumos14_rgb_test.py
new file mode 100644
index 0000000..5915889
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/localization/ssn/ssn_r50_450e_thumos14_rgb_test.py
@@ -0,0 +1,110 @@
+# model training and testing settings
+train_cfg = dict(
+ ssn=dict(
+ assigner=dict(
+ positive_iou_threshold=0.7,
+ background_iou_threshold=0.01,
+ incomplete_iou_threshold=0.3,
+ background_coverage_threshold=0.02,
+ incomplete_overlap_threshold=0.01),
+ sampler=dict(
+ num_per_video=8,
+ positive_ratio=1,
+ background_ratio=1,
+ incomplete_ratio=6,
+ add_gt_as_proposals=True),
+ loss_weight=dict(comp_loss_weight=0.1, reg_loss_weight=0.1),
+ debug=False))
+test_cfg = dict(
+ ssn=dict(
+ sampler=dict(test_interval=6, batch_size=16),
+ evaluater=dict(
+ top_k=2000,
+ nms=0.2,
+ softmax_before_filter=True,
+ cls_score_dict=None,
+ cls_top_k=2)))
+# model settings
+model = dict(
+ type='SSN',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ partial_bn=True),
+ spatial_type='avg',
+ dropout_ratio=0.8,
+ cls_head=dict(
+ type='SSNHead',
+ dropout_ratio=0.,
+ in_channels=2048,
+ num_classes=20,
+ consensus=dict(type='STPPTest', stpp_stage=(1, 1, 1)),
+ use_regression=True),
+ test_cfg=test_cfg)
+# dataset settings
+dataset_type = 'SSNDataset'
+data_root = './data/thumos14/rawframes/'
+data_root_val = './data/thumos14/rawframes/'
+ann_file_train = 'data/thumos14/thumos14_tag_val_proposal_list.txt'
+ann_file_val = 'data/thumos14/thumos14_tag_val_proposal_list.txt'
+ann_file_test = 'data/thumos14/thumos14_tag_test_proposal_list.txt'
+img_norm_cfg = dict(mean=[104, 117, 128], std=[1, 1, 1], to_bgr=True)
+test_pipeline = [
+ dict(
+ type='SampleProposalFrames',
+ clip_len=1,
+ body_segments=5,
+ aug_segments=(2, 2),
+ aug_ratio=0.5,
+ mode='test'),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(340, 256), keep_ratio=True),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(
+ type='Collect',
+ keys=[
+ 'imgs', 'relative_proposal_list', 'scale_factor_list',
+ 'proposal_tick_list', 'reg_norm_consts'
+ ],
+ meta_keys=[]),
+ dict(
+ type='ToTensor',
+ keys=[
+ 'imgs', 'relative_proposal_list', 'scale_factor_list',
+ 'proposal_tick_list', 'reg_norm_consts'
+ ])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root,
+ train_cfg=train_cfg,
+ test_cfg=test_cfg,
+ aug_ratio=0.5,
+ test_mode=True,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9,
+ weight_decay=1e-6) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[200, 400])
+checkpoint_config = dict(interval=5)
+log_config = dict(interval=5, hooks=[dict(type='TextLoggerHook')])
+# runtime settings
+total_epochs = 450
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/ssn_r50_1x5_450e_thumos14_rgb'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/localization/ssn/ssn_r50_450e_thumos14_rgb_train.py b/Downstream/Open-Set-Action-Recognition/configs/localization/ssn/ssn_r50_450e_thumos14_rgb_train.py
new file mode 100644
index 0000000..c64766c
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/localization/ssn/ssn_r50_450e_thumos14_rgb_train.py
@@ -0,0 +1,155 @@
+# model training and testing settings
+train_cfg = dict(
+ ssn=dict(
+ assigner=dict(
+ positive_iou_threshold=0.7,
+ background_iou_threshold=0.01,
+ incomplete_iou_threshold=0.3,
+ background_coverage_threshold=0.02,
+ incomplete_overlap_threshold=0.01),
+ sampler=dict(
+ num_per_video=8,
+ positive_ratio=1,
+ background_ratio=1,
+ incomplete_ratio=6,
+ add_gt_as_proposals=True),
+ loss_weight=dict(comp_loss_weight=0.1, reg_loss_weight=0.1),
+ debug=False))
+test_cfg = dict(
+ ssn=dict(
+ sampler=dict(test_interval=6, batch_size=16),
+ evaluater=dict(
+ top_k=2000,
+ nms=0.2,
+ softmax_before_filter=True,
+ cls_score_dict=None,
+ cls_top_k=2)))
+# model settings
+model = dict(
+ type='SSN',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ partial_bn=True),
+ spatial_type='avg',
+ dropout_ratio=0.8,
+ loss_cls=dict(type='SSNLoss'),
+ cls_head=dict(
+ type='SSNHead',
+ dropout_ratio=0.,
+ in_channels=2048,
+ num_classes=20,
+ consensus=dict(
+ type='STPPTrain',
+ stpp_stage=(1, 1, 1),
+ num_segments_list=(2, 5, 2)),
+ use_regression=True),
+ train_cfg=train_cfg)
+# dataset settings
+dataset_type = 'SSNDataset'
+data_root = './data/thumos14/rawframes/'
+data_root_val = './data/thumos14/rawframes/'
+ann_file_train = 'data/thumos14/thumos14_tag_val_proposal_list.txt'
+ann_file_val = 'data/thumos14/thumos14_tag_val_proposal_list.txt'
+ann_file_test = 'data/thumos14/thumos14_tag_test_proposal_list.txt'
+img_norm_cfg = dict(mean=[104, 117, 128], std=[1, 1, 1], to_bgr=True)
+train_pipeline = [
+ dict(
+ type='SampleProposalFrames',
+ clip_len=1,
+ body_segments=5,
+ aug_segments=(2, 2),
+ aug_ratio=0.5),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(340, 256), keep_ratio=True),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NPTCHW'),
+ dict(
+ type='Collect',
+ keys=[
+ 'imgs', 'reg_targets', 'proposal_scale_factor', 'proposal_labels',
+ 'proposal_type'
+ ],
+ meta_keys=[]),
+ dict(
+ type='ToTensor',
+ keys=[
+ 'imgs', 'reg_targets', 'proposal_scale_factor', 'proposal_labels',
+ 'proposal_type'
+ ])
+]
+val_pipeline = [
+ dict(
+ type='SampleProposalFrames',
+ clip_len=1,
+ body_segments=5,
+ aug_segments=(2, 2),
+ aug_ratio=0.5),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(340, 256), keep_ratio=True),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NPTCHW'),
+ dict(
+ type='Collect',
+ keys=[
+ 'imgs', 'reg_targets', 'proposal_scale_factor', 'proposal_labels',
+ 'proposal_type'
+ ],
+ meta_keys=[]),
+ dict(
+ type='ToTensor',
+ keys=[
+ 'imgs', 'reg_targets', 'proposal_scale_factor', 'proposal_labels',
+ 'proposal_type'
+ ])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ train_cfg=train_cfg,
+ test_cfg=test_cfg,
+ body_segments=5,
+ aug_segments=(2, 2),
+ aug_ratio=0.5,
+ test_mode=False,
+ verbose=True,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root,
+ train_cfg=train_cfg,
+ test_cfg=test_cfg,
+ body_segments=5,
+ aug_segments=(2, 2),
+ aug_ratio=0.5,
+ test_mode=False,
+ pipeline=val_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9,
+ weight_decay=1e-6) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[200, 400])
+checkpoint_config = dict(interval=5)
+log_config = dict(interval=1, hooks=[dict(type='TextLoggerHook')])
+# runtime settings
+total_epochs = 450
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/ssn_r50_1x5_450e_thumos14_rgb'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+find_unused_parameters = True
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/c3d/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/c3d/README.md
new file mode 100644
index 0000000..9c5c724
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/c3d/README.md
@@ -0,0 +1,71 @@
+# C3D
+
+## Introduction
+```
+@ARTICLE{2014arXiv1412.0767T,
+author = {Tran, Du and Bourdev, Lubomir and Fergus, Rob and Torresani, Lorenzo and Paluri, Manohar},
+title = "{Learning Spatiotemporal Features with 3D Convolutional Networks}",
+keywords = {Computer Science - Computer Vision and Pattern Recognition},
+year = 2014,
+month = dec,
+eid = {arXiv:1412.0767}
+}
+
+@article{Tran2014C3DGF,
+ title={C3D: Generic Features for Video Analysis},
+ author={D. Tran and Lubomir D. Bourdev and R. Fergus and L. Torresani and Manohar Paluri},
+ journal={ArXiv},
+ year={2014},
+ volume={abs/1412.0767}
+}
+```
+
+## Model Zoo
+
+### UCF-101
+
+| config | resolution | gpus | backbone | pretrain | top1 acc | top5 acc | testing protocol| inference_time(video/s) | gpu_mem(M) | ckpt | log | json |
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[c3d_sports1m_16x1x1_45e_ucf101_rgb.py](/configs/recognition/c3d/c3d_sports1m_16x1x1_45e_ucf101_rgb.py)|128x171|8| c3d | sports1m | 83.27 | 95.90 | 10 clips x 1 crop | x | 6053 | [ckpt](https://download.openmmlab.com/mmaction/recognition/c3d/c3d_sports1m_16x1x1_45e_ucf101_rgb/c3d_sports1m_16x1x1_45e_ucf101_rgb_20201021-26655025.pth)|[log](https://download.openmmlab.com/mmaction/recognition/c3d/c3d_sports1m_16x1x1_45e_ucf101_rgb/20201021_140429.log)|[json](https://download.openmmlab.com/mmaction/recognition/c3d/c3d_sports1m_16x1x1_45e_ucf101_rgb/20201021_140429.log.json)|
+
+Notes:
+
+1. The author of C3D normalized UCF-101 with volume mean and used SVM to classify videos, while we normalized the dataset with RGB mean value and used a linear classifier.
+2. The **gpus** indicates the number of gpu (32G V100) we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+3. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+
+For more details on data preparation, you can refer to UCF-101 in [Data Preparation](/docs/data_preparation.md).
+
+
+## Train
+
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train C3D model on UCF-101 dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/recognition/c3d/c3d_sports1m_16x1x1_45e_ucf101_rgb.py \
+ --validate --seed 0 --deterministic
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test C3D model on UCF-101 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/c3d/c3d_sports1m_16x1x1_45e_ucf101_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/c3d/c3d_sports1m_16x1x1_45e_ucf101_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/c3d/c3d_sports1m_16x1x1_45e_ucf101_rgb.py
new file mode 100644
index 0000000..1b05441
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/c3d/c3d_sports1m_16x1x1_45e_ucf101_rgb.py
@@ -0,0 +1,118 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='C3D',
+ pretrained= # noqa: E251
+ 'https://download.openmmlab.com/mmaction/recognition/c3d/c3d_sports1m_pretrain_20201016-dcc47ddc.pth', # noqa: E501
+ style='pytorch',
+ conv_cfg=dict(type='Conv3d'),
+ norm_cfg=None,
+ act_cfg=dict(type='ReLU'),
+ dropout_ratio=0.5,
+ init_std=0.005),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=101,
+ in_channels=4096,
+ spatial_type=None,
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='score')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/ucf101/rawframes'
+data_root_val = 'data/ucf101/rawframes'
+split = 1 # official train/test splits. valid numbers: 1, 2, 3
+ann_file_train = f'data/ucf101/ucf101_train_split_{split}_rawframes.txt'
+ann_file_val = f'data/ucf101/ucf101_val_split_{split}_rawframes.txt'
+ann_file_test = f'data/ucf101/ucf101_val_split_{split}_rawframes.txt'
+img_norm_cfg = dict(mean=[104, 117, 128], std=[1, 1, 1], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=16, frame_interval=1, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(128, 171)),
+ dict(type='RandomCrop', size=112),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=16,
+ frame_interval=1,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(128, 171)),
+ dict(type='CenterCrop', crop_size=112),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=16,
+ frame_interval=1,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(128, 171)),
+ dict(type='CenterCrop', crop_size=112),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+data = dict(
+ videos_per_gpu=30,
+ workers_per_gpu=2,
+ test_dataloader=dict(videos_per_gpu=1),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9,
+ weight_decay=0.0005) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 45
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = f'./work_dirs/c3d_sports1m_16x1x1_45e_ucf101_split_{split}_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/README.md
new file mode 100644
index 0000000..55185fe
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/README.md
@@ -0,0 +1,73 @@
+# CSN
+
+## Introduction
+```
+@inproceedings{inproceedings,
+author = {Wang, Heng and Feiszli, Matt and Torresani, Lorenzo},
+year = {2019},
+month = {10},
+pages = {5551-5560},
+title = {Video Classification With Channel-Separated Convolutional Networks},
+doi = {10.1109/ICCV.2019.00565}
+}
+
+@inproceedings{ghadiyaram2019large,
+ title={Large-scale weakly-supervised pre-training for video action recognition},
+ author={Ghadiyaram, Deepti and Tran, Du and Mahajan, Dhruv},
+ booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
+ pages={12046--12055},
+ year={2019}
+}
+```
+
+## Model Zoo
+
+### Kinetics-400
+
+|config | resolution | gpus | backbone |pretrain| top1 acc| top5 acc | inference_time(video/s) | gpu_mem(M)| ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb.py](/configs/recognition/csn/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb.py)|short-side 320|8x4| ResNet152 | IG65M|80.14|94.93|x|8517|[ckpt](https://download.openmmlab.com/mmaction/recognition/csn/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb_20200803-fc66ce8d.pth)|[log](https://download.openmmlab.com/mmaction/recognition/csn/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb/20200728_031952.log)|[json](https://download.openmmlab.com/mmaction/recognition/csn/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb/20200728_031952.log.json)|
+|[ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb.py](/configs/recognition/csn/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb.py)|short-side 320|8x4| ResNet152 | IG65M|82.76|95.68|x|8516|[ckpt](https://download.openmmlab.com/mmaction/recognition/csn/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb_20200812-9037a758.pth)|[log](https://download.openmmlab.com/mmaction/recognition/csn/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb/20200809_053132.log)|[json](https://download.openmmlab.com/mmaction/recognition/csn/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb/20200809_053132.log.json)|
+
+Notes:
+
+1. The **gpus** indicates the number of gpu (32G V100) we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+3. The values in columns named after "reference" are the results got by training on the original repo, using the same model settings.
+
+For more details on data preparation, you can refer to Kinetics400 in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train CSN model on Kinetics-400 dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/recognition/csn/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb.py \
+ --work-dir work_dirs/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb \
+ --validate --seed 0 --deterministic
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test CSN model on Kinetics-400 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/csn/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json --average-clips prob
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/finetune_ucf101_csn_dnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/finetune_ucf101_csn_dnn.py
new file mode 100644
index 0000000..dd67a53
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/finetune_ucf101_csn_dnn.py
@@ -0,0 +1,127 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dCSN',
+ pretrained2d=False,
+ pretrained= # noqa: E251
+ 'https://download.openmmlab.com/mmaction/recognition/csn/ircsn_from_scratch_r152_ig65m_20200807-771c4135.pth', # noqa: E501
+ depth=152,
+ with_pool2=False,
+ bottleneck_mode='ir',
+ norm_eval=True,
+ bn_frozen=True,
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.000125, momentum=0.9,
+ weight_decay=0.0001, nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='step',
+ step=[20, 40],
+ warmup='linear',
+ warmup_ratio=0.1,
+ warmup_by_epoch=True,
+ warmup_iters=5)
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook')])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/csn/finetune_ucf101_csn_dnn' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/csn/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb_20200812-9037a758.pth'
+resume_from = None
+workflow = [('train', 1)]
+find_unused_parameters = True
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/finetune_ucf101_csn_edlnokl_avuc_debias.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/finetune_ucf101_csn_edlnokl_avuc_debias.py
new file mode 100644
index 0000000..6d2268f
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/finetune_ucf101_csn_edlnokl_avuc_debias.py
@@ -0,0 +1,144 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dCSN',
+ pretrained2d=False,
+ pretrained= # noqa: E251
+ 'https://download.openmmlab.com/mmaction/recognition/csn/ircsn_from_scratch_r152_ig65m_20200807-771c4135.pth', # noqa: E501
+ depth=152,
+ with_pool2=False,
+ bottleneck_mode='ir',
+ norm_eval=True,
+ bn_frozen=True,
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01),
+ debias_head=dict(
+ type='DebiasHead',
+ loss_cls=evidence_loss, # actually not used!
+ loss_factor=0.1,
+ num_classes=101,
+ in_channels=2048,
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.000125, momentum=0.9,
+ weight_decay=0.0001, nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='step',
+ step=[20, 40],
+ warmup='linear',
+ warmup_ratio=0.1,
+ warmup_by_epoch=True,
+ warmup_iters=5)
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook')])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/csn/finetune_ucf101_csn_edlnokl_avuc_debias' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/csn/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb_20200812-9037a758.pth'
+resume_from = None
+workflow = [('train', 1)]
+find_unused_parameters = True
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/inference_csn_dnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/inference_csn_dnn.py
new file mode 100644
index 0000000..e442efb
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/inference_csn_dnn.py
@@ -0,0 +1,52 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dCSN',
+ pretrained2d=False,
+ pretrained= # noqa: E251
+ 'https://download.openmmlab.com/mmaction/recognition/csn/ircsn_from_scratch_r152_ig65m_20200807-771c4135.pth', # noqa: E501
+ depth=152,
+ with_pool2=False,
+ bottleneck_mode='ir',
+ norm_eval=True,
+ bn_frozen=True,
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/inference_csn_enn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/inference_csn_enn.py
new file mode 100644
index 0000000..439e755
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/inference_csn_enn.py
@@ -0,0 +1,58 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dCSN',
+ pretrained2d=False,
+ pretrained= # noqa: E251
+ 'https://download.openmmlab.com/mmaction/recognition/csn/ircsn_from_scratch_r152_ig65m_20200807-771c4135.pth', # noqa: E501
+ depth=152,
+ with_pool2=False,
+ bottleneck_mode='ir',
+ norm_eval=True,
+ bn_frozen=True,
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ loss_cls=dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ annealing_method='exp'),
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+evidence='exp' # only used for EDL
+test_cfg = dict(average_clips='score')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb.py
new file mode 100644
index 0000000..d833fa5
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb.py
@@ -0,0 +1,124 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dCSN',
+ pretrained2d=False,
+ pretrained= # noqa: E251
+ 'https://download.openmmlab.com/mmaction/recognition/csn/ircsn_from_scratch_r152_ig65m_20200807-771c4135.pth', # noqa: E501
+ depth=152,
+ with_pool2=False,
+ bottleneck_mode='ir',
+ norm_eval=True,
+ bn_frozen=True,
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=3,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.000125, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='step',
+ step=[32, 48],
+ warmup='linear',
+ warmup_ratio=0.1,
+ warmup_by_epoch=True,
+ warmup_iters=16)
+total_epochs = 58
+checkpoint_config = dict(interval=2)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook')])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/ircsn_ig65m_pretrained_bnfrozen_r152_32x2x1_58e_kinetics400_rgb' # noqa: E501
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+find_unused_parameters = True
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb.py
new file mode 100644
index 0000000..76f99e0
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/csn/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb.py
@@ -0,0 +1,122 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dCSN',
+ pretrained2d=False,
+ pretrained= # noqa: E251
+ 'https://download.openmmlab.com/mmaction/recognition/csn/ircsn_from_scratch_r152_ig65m_20200807-771c4135.pth', # noqa: E501
+ depth=152,
+ with_pool2=False,
+ bottleneck_mode='ir',
+ norm_eval=False,
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=3,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.000125, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='step',
+ step=[32, 48],
+ warmup='linear',
+ warmup_ratio=0.1,
+ warmup_by_epoch=True,
+ warmup_iters=16)
+total_epochs = 58
+checkpoint_config = dict(interval=2)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook')])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/ircsn_ig65m_pretrained_r152_32x2x1_58e_kinetics400_rgb'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/README.md
new file mode 100644
index 0000000..a98d62d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/README.md
@@ -0,0 +1,76 @@
+# I3D
+
+## Introduction
+```
+@inproceedings{inproceedings,
+ author = {Carreira, J. and Zisserman, Andrew},
+ year = {2017},
+ month = {07},
+ pages = {4724-4733},
+ title = {Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset},
+ doi = {10.1109/CVPR.2017.502}
+}
+
+@article{NonLocal2018,
+ author = {Xiaolong Wang and Ross Girshick and Abhinav Gupta and Kaiming He},
+ title = {Non-local Neural Networks},
+ journal = {CVPR},
+ year = {2018}
+}
+```
+
+## Model Zoo
+
+### Kinetics-400
+
+|config | resolution | gpus | backbone |pretrain| top1 acc| top5 acc | inference_time(video/s) | gpu_mem(M)| ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[i3d_r50_32x2x1_100e_kinetics400_rgb](/configs/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb.py) |340x256|8| ResNet50|ImageNet |72.68|90.78|1.7 (320x3 frames)| 5170|[ckpt](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb/i3d_r50_32x2x1_100e_kinetics400_rgb_20200614-c25ef9a4.pth) | [log](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb/20200614_060456.log)| [json](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb/20200614_060456.log.json)|
+|[i3d_r50_32x2x1_100e_kinetics400_rgb](/configs/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb.py) |short-side 256|8| ResNet50|ImageNet | 73.27|90.92|x|5170|[ckpt](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_256p_32x2x1_100e_kinetics400_rgb/i3d_r50_256p_32x2x1_100e_kinetics400_rgb_20200801-7d9f44de.pth)|[log](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_256p_32x2x1_100e_kinetics400_rgb/20200725_031555.log)|[json](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_256p_32x2x1_100e_kinetics400_rgb/20200725_031555.log.json)|
+|[i3d_r50_video_32x2x1_100e_kinetics400_rgb](/configs/recognition/i3d/i3d_r50_video_32x2x1_100e_kinetics400_rgb.py)|short-side 256p|8| ResNet50 |ImageNet|72.85 |90.75 |x|5170|[ckpt](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_video_32x2x1_100e_kinetics400_rgb/i3d_r50_video_32x2x1_100e_kinetics400_rgb_20200826-e31c6f52.pth)|[log](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_video_32x2x1_100e_kinetics400_rgb/20200706_143014.log)|[json](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_video_32x2x1_100e_kinetics400_rgb/20200706_143014.log.json)|
+|[i3d_r50_dense_32x2x1_100e_kinetics400_rgb](/configs/recognition/i3d/i3d_r50_dense_32x2x1_100e_kinetics400_rgb.py) |340x256|8x2| ResNet50| ImageNet|72.77|90.57|1.7 (320x3 frames)| 5170| [ckpt](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_32x2x1_100e_kinetics400_rgb/i3d_r50_dense_32x2x1_100e_kinetics400_rgb_20200616-2bbb4361.pth) | [log](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_32x2x1_100e_kinetics400_rgb/20200616_230011.log)| [json](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_32x2x1_100e_kinetics400_rgb/20200616_230011.log.json)|
+|[i3d_r50_dense_32x2x1_100e_kinetics400_rgb](/configs/recognition/i3d/i3d_r50_dense_32x2x1_100e_kinetics400_rgb.py) |short-side 256|8| ResNet50| ImageNet|73.48|91.00|x| 5170| [ckpt](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb_20200725-24eb54cc.pth)|[log](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb/20200725_031604.log)|[json](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb/20200725_031604.log.json)|
+|[i3d_r50_lazy_32x2x1_100e_kinetics400_rgb](/configs/recognition/i3d/i3d_r50_lazy_32x2x1_100e_kinetics400_rgb.py) |340x256|8| ResNet50 |ImageNet|72.32|90.72|1.8 (320x3 frames)| 5170| [ckpt](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_fast_32x2x1_100e_kinetics400_rgb/i3d_r50_fast_32x2x1_100e_kinetics400_rgb_20200612-000e4d2a.pth) | [log](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_fast_32x2x1_100e_kinetics400_rgb/20200612_233836.log)| [json](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_fast_32x2x1_100e_kinetics400_rgb/20200612_233836.log.json)|
+|[i3d_r50_lazy_32x2x1_100e_kinetics400_rgb](/configs/recognition/i3d/i3d_r50_lazy_32x2x1_100e_kinetics400_rgb.py) |short-side 256|8| ResNet50| ImageNet|73.24|90.99|x| 5170| [ckpt](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_fast_256p_32x2x1_100e_kinetics400_rgb/i3d_r50_fast_256p_32x2x1_100e_kinetics400_rgb_20200817-4e90d1d5.pth)| [log](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_fast_256p_32x2x1_100e_kinetics400_rgb/20200725_031457.log) | [json](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_fast_256p_32x2x1_100e_kinetics400_rgb/20200725_031457.log.json) |
+|[i3d_nl_embedded_gaussian_r50_32x2x1_100e_kinetics400_rgb](/configs/recognition/i3d/i3d_nl_embedded_gaussian_r50_32x2x1_100e_kinetics400_rgb.py)|short-side 256p|8x4| ResNet50 |ImageNet|74.71|91.81|x|6438|[ckpt](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_nl_embedded_gaussian_r50_32x2x1_100e_kinetics400_rgb/i3d_nl_embedded_gaussian_r50_32x2x1_100e_kinetics400_rgb_20200813-6e6aef1b.pth)|[log](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_nl_embedded_gaussian_r50_32x2x1_100e_kinetics400_rgb/20200813_034054.log)|[json](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_nl_embedded_gaussian_r50_32x2x1_100e_kinetics400_rgb/20200813_034054.log.json)|
+|[i3d_nl_gaussian_r50_32x2x1_100e_kinetics400_rgb](/configs/recognition/i3d/i3d_nl_gaussian_r50_32x2x1_100e_kinetics400_rgb.py)|short-side 256p|8x4| ResNet50 |ImageNet|73.37|91.26|x|4944|[ckpt](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_nl_gaussian_r50_32x2x1_100e_kinetics400_rgb/i3d_nl_gaussian_r50_32x2x1_100e_kinetics400_rgb_20200815-17f84aa2.pth)|[log](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_nl_gaussian_r50_32x2x1_100e_kinetics400_rgb/20200813_034909.log)|[json](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_nl_gaussian_r50_32x2x1_100e_kinetics400_rgb/20200813_034909.log.json)|
+|[i3d_nl_dot_product_r50_32x2x1_100e_kinetics400_rgb](/configs/recognition/i3d/i3d_nl_dot_product_r50_32x2x1_100e_kinetics400_rgb.py)|short-side 256p|8x4| ResNet50 |ImageNet|73.92|91.59|x|4832|[ckpt](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_nl_dot_product_r50_32x2x1_100e_kinetics400_rgb/i3d_nl_dot_product_r50_32x2x1_100e_kinetics400_rgb_20200814-7c30d5bb.pth)|[log](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_nl_dot_product_r50_32x2x1_100e_kinetics400_rgb/20200814_044208.log)|[json](https://download.openmmlab.com/mmaction/recognition/i3d/i3d_nl_dot_product_r50_32x2x1_100e_kinetics400_rgb/20200814_044208.log.json)|
+
+Notes:
+1. The **gpus** indicates the number of gpu we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+
+For more details on data preparation, you can refer to Kinetics400 in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train I3D model on Kinetics-400 dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb.py \
+ --work-dir work_dirs/i3d_r50_32x2x1_100e_kinetics400_rgb \
+ --validate --seed 0 --deterministic
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test I3D model on Kinetics-400 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json --average-clips prob
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_bnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_bnn.py
new file mode 100644
index 0000000..62f3d6a
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_bnn.py
@@ -0,0 +1,128 @@
+# model settings
+model = dict(
+ type='Recognizer3DBNN',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DBNNHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = dict(loss_weight=1e-6, npass=2)
+test_cfg = dict(average_clips='prob', npass=10)
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8, # set to 8 for training
+ workers_per_gpu=4, # set to 4 for training
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ start_index=0,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, # change from 0.01 to 0.001
+ weight_decay=0.0001, nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50 # change from 100 to 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/finetune_ucf101_i3d_bnn/'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb_20200725-24eb54cc.pth' # model path can be found in model zoo
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_dnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_dnn.py
new file mode 100644
index 0000000..92859fc
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_dnn.py
@@ -0,0 +1,128 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8, # set to 8 for training
+ workers_per_gpu=4, # set to 4 for training
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ start_index=0,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, # change from 0.01 to 0.001
+ weight_decay=0.0001, nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50 # change from 100 to 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/finetune_ucf101_i3d_dnn/'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb_20200725-24eb54cc.pth' # model path can be found in model zoo
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_edlnokl.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_edlnokl.py
new file mode 100644
index 0000000..f2157f4
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_edlnokl.py
@@ -0,0 +1,137 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=False,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8, # set to 2 for evaluation on GPU with 24GB
+ workers_per_gpu=4, # set to 2 for evaluation on GPU with 24GB
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ start_index=0,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, # change from 0.01 to 0.001
+ weight_decay=0.0001, nesterov=True)
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40]) # change from [40,80] to [20,40]
+total_epochs = 50 # change from 100 to 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/finetune_ucf101_i3d_edlnokl/'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb_20200725-24eb54cc.pth' # model path can be found in model zoo
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_edlnokl_avuc_debias.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_edlnokl_avuc_debias.py
new file mode 100644
index 0000000..d166647
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_edlnokl_avuc_debias.py
@@ -0,0 +1,145 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01),
+ debias_head=dict(
+ type='DebiasHead',
+ loss_cls=evidence_loss, # actually not used!
+ loss_factor=0.1,
+ num_classes=101,
+ in_channels=2048,
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8, # set to 2 for evaluation on GPU with 24GB
+ workers_per_gpu=4, # set to 2 for evaluation on GPU with 24GB
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ start_index=0,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, # change from 0.01 to 0.001
+ weight_decay=0.0001, nesterov=True)
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40]) # change from [40,80] to [20,40]
+total_epochs = 50 # change from 100 to 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/finetune_ucf101_i3d_edlnokl_avuc_debias/'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb_20200725-24eb54cc.pth' # model path can be found in model zoo
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_rpl.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_rpl.py
new file mode 100644
index 0000000..e96d7f7
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/finetune_ucf101_i3d_rpl.py
@@ -0,0 +1,131 @@
+# model settings
+model = dict(
+ type='Recognizer3DRPL',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DRPLHead',
+ loss_cls=dict(type='RPLoss',
+ temperature=1,
+ weight_pl=0.1),
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8, # set to 8 for training
+ workers_per_gpu=4, # set to 4 for training
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ start_index=0,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, # change from 0.01 to 0.001
+ weight_decay=0.0001, nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50 # change from 100 to 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/finetune_ucf101_i3d_rpl/'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb_20200725-24eb54cc.pth' # model path can be found in model zoo
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_nl_dot_product_r50_32x2x1_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_nl_dot_product_r50_32x2x1_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..878bc7d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_nl_dot_product_r50_32x2x1_100e_kinetics400_rgb.py
@@ -0,0 +1,128 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ non_local=((0, 0, 0), (0, 1, 0, 1), (0, 1, 0, 1, 0, 1), (0, 0, 0)),
+ non_local_cfg=dict(
+ sub_sample=True,
+ use_scale=False,
+ norm_cfg=dict(type='BN3d', requires_grad=True),
+ mode='dot_product'),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/i3d_nl_dot_product_r50_32x2x1_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_nl_embedded_gaussian_r50_32x2x1_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_nl_embedded_gaussian_r50_32x2x1_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..73454b4
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_nl_embedded_gaussian_r50_32x2x1_100e_kinetics400_rgb.py
@@ -0,0 +1,128 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ non_local=((0, 0, 0), (0, 1, 0, 1), (0, 1, 0, 1, 0, 1), (0, 0, 0)),
+ non_local_cfg=dict(
+ sub_sample=True,
+ use_scale=False,
+ norm_cfg=dict(type='BN3d', requires_grad=True),
+ mode='embedded_gaussian'),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/i3d_nl_embedded_gaussian_r50_32x2x1_100e_kinetics400_rgb/' # noqa: E501
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_nl_gaussian_r50_32x2x1_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_nl_gaussian_r50_32x2x1_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..5071f98
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_nl_gaussian_r50_32x2x1_100e_kinetics400_rgb.py
@@ -0,0 +1,128 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ non_local=((0, 0, 0), (0, 1, 0, 1), (0, 1, 0, 1, 0, 1), (0, 0, 0)),
+ non_local_cfg=dict(
+ sub_sample=True,
+ use_scale=False,
+ norm_cfg=dict(type='BN3d', requires_grad=True),
+ mode='gaussian'),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/i3d_nl_gaussian_r50_32x2x1_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..5d279f2
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb.py
@@ -0,0 +1,122 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/i3d_r50_32x2x1_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_dense_32x2x1_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_dense_32x2x1_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..962ca81
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_dense_32x2x1_100e_kinetics400_rgb.py
@@ -0,0 +1,122 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DenseSampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/i3d_r50_dense_32x2x1_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_heavy_8x8x1_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_heavy_8x8x1_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..38fa186
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_heavy_8x8x1_100e_kinetics400_rgb.py
@@ -0,0 +1,125 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ with_pool2=True,
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/i3d_r50_heavy_8x8x1_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_lazy_32x2x1_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_lazy_32x2x1_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..d6d969d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_lazy_32x2x1_100e_kinetics400_rgb.py
@@ -0,0 +1,122 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='RawFrameDecode', decoding_backend='turbojpeg'),
+ dict(type='Resize', scale=(-1, 256), lazy=True),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0,
+ lazy=True),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False, lazy=True),
+ dict(type='Flip', flip_ratio=0.5, lazy=True),
+ dict(type='Fuse'),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode', decoding_backend='turbojpeg'),
+ dict(type='Resize', scale=(-1, 256), lazy=True),
+ dict(type='CenterCrop', crop_size=224, lazy=True),
+ dict(type='Flip', flip_ratio=0, lazy=True),
+ dict(type='Fuse'),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode', decoding_backend='turbojpeg'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/i3d_r50_lazy_32x2x1_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_video_32x2x1_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_video_32x2x1_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..1a27f2f
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_video_32x2x1_100e_kinetics400_rgb.py
@@ -0,0 +1,125 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics400/videos_train'
+data_root_val = 'data/kinetics400/videos_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_videos.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_videos.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/i3d_r50_video_3d_32x2x1_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_video_heavy_8x8x1_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_video_heavy_8x8x1_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..11ee7de
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_video_heavy_8x8x1_100e_kinetics400_rgb.py
@@ -0,0 +1,128 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ with_pool2=True,
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics400/videos_train'
+data_root_val = 'data/kinetics400/videos_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_videos.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_videos.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/i3d_r50_video_heavy_8x8x1_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_video_inference_32x2x1_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_video_inference_32x2x1_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..9e5e83a
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/i3d_r50_video_inference_32x2x1_100e_kinetics400_rgb.py
@@ -0,0 +1,49 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='DecordInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_bnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_bnn.py
new file mode 100644
index 0000000..df9c473
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_bnn.py
@@ -0,0 +1,49 @@
+# model settings
+model = dict(
+ type='Recognizer3DBNN',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DBNNHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0,
+ init_std=0.01))
+# model training and testing settings
+test_cfg = dict(average_clips='prob', npass=10)
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_dnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_dnn.py
new file mode 100644
index 0000000..f7c93e1
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_dnn.py
@@ -0,0 +1,49 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_enn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_enn.py
new file mode 100644
index 0000000..034eda2
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_enn.py
@@ -0,0 +1,58 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ # pretrained='work_dirs/i3d/finetune_ucf101_i3d_edlnokl/latest.pth',
+ # pretrained=False,
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ loss_cls=dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ annealing_method='exp'),
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+evidence='exp' # only used for EDL
+test_cfg = dict(average_clips='score')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
+dist_params = dict(backend='nccl')
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_rpl.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_rpl.py
new file mode 100644
index 0000000..f364ee4
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/inference_i3d_rpl.py
@@ -0,0 +1,52 @@
+# model settings
+model = dict(
+ type='Recognizer3DRPL',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DRPLHead',
+ loss_cls=dict(type='RPLoss',
+ temperature=1,
+ weight_pl=0.1),
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/train_kinetics10_i3d_DEAR.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/train_kinetics10_i3d_DEAR.py
new file mode 100644
index 0000000..4d6cd68
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/train_kinetics10_i3d_DEAR.py
@@ -0,0 +1,145 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=10,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ loss_cls=evidence_loss,
+ num_classes=10,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01),
+ debias_head=dict(
+ type='DebiasHead',
+ loss_cls=evidence_loss, # actually not used!
+ loss_factor=0.1,
+ num_classes=10,
+ in_channels=2048,
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics10/videos_train'
+data_root_val = 'data/kinetics10/videos_val'
+ann_file_train = 'data/kinetics10/kinetics10_train_list_videos.txt'
+ann_file_val = 'data/kinetics10/kinetics10_val_list_videos.txt'
+ann_file_test = 'data/kinetics10/kinetics10_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8, # set to 2 for evaluation on GPU with 24GB
+ workers_per_gpu=4, # set to 2 for evaluation on GPU with 24GB
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ start_index=0,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, # change from 0.01 to 0.001
+ weight_decay=0.0001, nesterov=True)
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40]) # change from [40,80] to [20,40]
+total_epochs = 50 # change from 100 to 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/train_kinetics10_i3d_DEAR/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/train_kinetics10_i3d_DEAR_noDebias.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/train_kinetics10_i3d_DEAR_noDebias.py
new file mode 100644
index 0000000..d4f2700
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/i3d/train_kinetics10_i3d_DEAR_noDebias.py
@@ -0,0 +1,137 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=10,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3d',
+ pretrained2d=True,
+ pretrained='torchvision://resnet50',
+ depth=50,
+ conv_cfg=dict(type='Conv3d'),
+ norm_eval=False,
+ inflate=((1, 1, 1), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 1, 0)),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ loss_cls=evidence_loss,
+ num_classes=10,
+ in_channels=2048,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics10/videos_train'
+data_root_val = 'data/kinetics10/videos_val'
+ann_file_train = 'data/kinetics10/kinetics10_train_list_videos.txt'
+ann_file_val = 'data/kinetics10/kinetics10_val_list_videos.txt'
+ann_file_test = 'data/kinetics10/kinetics10_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8, # set to 2 for evaluation on GPU with 24GB
+ workers_per_gpu=4, # set to 2 for evaluation on GPU with 24GB
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ start_index=0,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, # change from 0.01 to 0.001
+ weight_decay=0.0001, nesterov=True)
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40]) # change from [40,80] to [20,40]
+total_epochs = 50 # change from 100 to 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/train_kinetics10_i3d_DEAR_noDebias/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/mae/finetune_ucf101_mae_edlnokl.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/mae/finetune_ucf101_mae_edlnokl.py
new file mode 100644
index 0000000..4475381
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/mae/finetune_ucf101_mae_edlnokl.py
@@ -0,0 +1,177 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=False,
+ annealing_method='exp')
+
+# mae huge ------------
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='VisionTransformer3D',
+ patch_size=16,
+ embed_dim=1280,
+ depth=32,
+ num_heads=16,
+ mlp_ratio=4,
+ qkv_bias=True,
+ num_classes=0,
+ pretrained='work_dirs/mae/finetune_ucf101_mae_dnn/huangbingkun/model/vit_h_hybridv2_pt_1200e_k700_ft_rep_2.pth'
+ ),
+ cls_head=dict(
+ type='BaseClsHead',
+ loss_cls=evidence_loss,
+ in_channels=1280,
+ num_classes=101,
+ dropout_ratio=0.5,
+ ))
+
+
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ # dict(type='OpenCVInit', num_threads=1),
+ dict(type='DecordInit'),
+ dict(type='DenseSampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ # dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=32),
+ # dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ # dict(type='OpenCVDecode'),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ # dict(type='OpenCVInit', num_threads=1),
+ dict(type='DecordInit'),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ # dict(
+ # type='SampleFrames',
+ # clip_len=1,
+ # frame_interval=1,
+ # num_clips=32,
+ # test_mode=True),
+ # dict(
+ # type='SampleFrames',
+ # clip_len=32,
+ # frame_interval=2,
+ # num_clips=1,
+ # test_mode=True),
+ # dict(type='OpenCVDecode'),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ # dict(type='OpenCVInit', num_threads=1),
+ dict(type='DecordInit'),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ # dict(
+ # type='SampleFrames',
+ # clip_len=1,
+ # frame_interval=1,
+ # num_clips=32,
+ # test_mode=True),
+ # dict(
+ # type='SampleFrames',
+ # clip_len=32,
+ # frame_interval=2,
+ # num_clips=1,
+ # test_mode=True),
+ # dict(type='OpenCVDecode'),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 224)),
+ dict(type='ThreeCrop', crop_size=224),
+ # dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=1, # set to 2 for evaluation on GPU with 24GB
+ workers_per_gpu=4, # set to 2 for evaluation on GPU with 24GB
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ start_index=0,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, # change from 0.01 to 0.001
+ weight_decay=0.0001, nesterov=True)
+
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40]) # change from [40,80] to [20,40]
+total_epochs = 50 # change from 100 to 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=60, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/finetune_ucf101_mae_edlnokl/'
+# load_from = 'https://download.openmmlab.com/mmaction/recognition/i3d/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb/i3d_r50_dense_256p_32x2x1_100e_kinetics400_rgb_20200725-24eb54cc.pth' # model path can be found in model zoo
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/mae/inference_mae_enn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/mae/inference_mae_enn.py
new file mode 100644
index 0000000..2b03c7c
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/mae/inference_mae_enn.py
@@ -0,0 +1,71 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=False,
+ annealing_method='exp')
+
+# mae huge
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='VisionTransformer3D',
+ patch_size=16,
+ embed_dim=1280,
+ depth=32,
+ num_heads=16,
+ mlp_ratio=4,
+ qkv_bias=True,
+ num_classes=0,
+ # pretrained='work_dirs/mae/finetune_ucf101_mae_dnn/huangbingkun/model/vit_h_hybridv2_pt_1200e_k700_ft_rep_2.pth'
+ ),
+ cls_head=dict(
+ type='BaseClsHead',
+ loss_cls=evidence_loss,
+ in_channels=1280,
+ num_classes=101,
+ dropout_ratio=0.5,
+ ))
+
+# model training and testing settings
+evidence='exp' # only used for EDL
+test_cfg = dict(average_clips='score')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ # dict(
+ # type='SampleFrames',
+ # clip_len=1,
+ # frame_interval=1,
+ # num_clips=32,
+ # test_mode=True),
+ # dict(type='OpenCVDecode'),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 224)),
+ dict(type='ThreeCrop', crop_size=224),
+ # dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=4,
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
+dist_params = dict(backend='nccl')
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/README.md
new file mode 100644
index 0000000..00ea48d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/README.md
@@ -0,0 +1,116 @@
+# Omni-sourced Webly-supervised Learning for Video Recognition
+
+[Haodong Duan](https://github.com/kennymckormick), [Yue Zhao](https://github.com/zhaoyue-zephyrus), [Yuanjun Xiong](https://github.com/yjxiong), Wentao Liu, [Dahua Lin](https://github.com/lindahua)
+
+In ECCV, 2020. [Paper](https://arxiv.org/abs/2003.13042)
+
+![pipeline](pipeline.png)
+
+### Kinetics-400 Model Release
+
+We currently released 4 models trained with OmniSource framework, including both 2D and 3D architectures. We compare the performance of models trained with or without OmniSource in the following table.
+
+| Model | Modality | Pretrained | Backbone | Input | Resolution | Top-1 (Baseline / OmniSource (Delta)) | Top-5 (Baseline / OmniSource (Delta))) | Download |
+| :------: | :------: | :--------: | :-------: | :---: | :------------: | :-----------------------------------: | :------------------------------------: | :----------------------------------------------------------: |
+| TSN | RGB | ImageNet | ResNet50 | 3seg | 340x256 | 70.6 / 73.6 (+ 3.0) | 89.4 / 91.0 (+ 1.6) | [Baseline](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth) / [OmniSource](https://download.openmmlab.com/mmaction/recognition/tsn/omni/tsn_imagenet_pretrained_r50_omni_1x1x3_kinetics400_rgb_20200926-54192355.pth) |
+| TSN | RGB | IG-1B | ResNet50 | 3seg | short-side 320 | 73.1 / 75.7 (+ 2.6) | 90.4 / 91.9 (+ 1.5) | [Baseline](https://download.openmmlab.com/mmaction/recognition/tsn/omni/tsn_1G1B_pretrained_r50_without_omni_1x1x3_kinetics400_rgb_20200926-c133dd49.pth) / [OmniSource](https://download.openmmlab.com/mmaction/recognition/tsn/omni/tsn_1G1B_pretrained_r50_omni_1x1x3_kinetics400_rgb_20200926-2863fed0.pth) |
+| SlowOnly | RGB | Scratch | ResNet50 | 4x16 | short-side 320 | 72.9 / 76.8 (+ 3.9) | 90.9 / 92.5 (+ 1.6) | [Baseline](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb/slowonly_r50_4x16x1_256e_kinetics400_rgb_20200704-a69556c6.pth) / [OmniSource](https://download.openmmlab.com/mmaction/recognition/slowonly/omni/slowonly_r50_omni_4x16x1_kinetics400_rgb_20200926-51b1f7ea.pth) |
+| SlowOnly | RGB | Scratch | ResNet101 | 8x8 | short-side 320 | 76.5 / 80.4 (+ 3.9) | 92.7 / 94.4 (+ 1.7) | [Baseline](https://download.openmmlab.com/mmaction/recognition/slowonly/omni/slowonly_r101_without_omni_8x8x1_kinetics400_rgb_20200926-0c730aef.pth) / [OmniSource](https://download.openmmlab.com/mmaction/recognition/slowonly/omni/slowonly_r101_omni_8x8x1_kinetics400_rgb_20200926-b5dbb701.pth) |
+
+### Benchmark on Mini-Kinetics
+
+We release a subset of web dataset used in the OmniSource paper. Specifically, we release the web data in the 200 classes of [Mini-Kinetics](https://arxiv.org/pdf/1712.04851.pdf). The statistics of those datasets is detailed in [preparing_omnisource](/tools/data/omnisource/README.md). To obtain those data, you need to fill in a [data request form](https://docs.google.com/forms/d/e/1FAIpQLSd8_GlmHzG8FcDbW-OEu__G7qLgOSYZpH-i5vYVJcu7wcb_TQ/viewform?usp=sf_link). After we received your request, the download link of these data will be send to you. For more details on the released OmniSource web dataset, please refer to [preparing_omnisource](/tools/data/omnisource/README.md).
+
+We benchmark the OmniSource framework on the released subset, results are listed in the following table (we report the Top-1 and Top-5 accuracy on Mini-Kinetics validation). The cbenchmark can be used as a baseline for video recognition with web data.
+
+
+
+We also list the benchmark in the original paper which run on Kinetics-400 for comparison:
+
+| Model | Baseline | +GG-img | +[GG-IG]-img | +IG-vid | +KRaw | OmniSource |
+| :--------------------: | :---------: | :---------: | :----------: | :---------: | :---------: | :---------: |
+| TSN-3seg-ResNet50 | 70.6 / 89.4 | 71.5 / 89.5 | 72.0 / 90.0 | 72.0 / 90.3 | 71.7 / 89.6 | 73.6 / 91.0 |
+| SlowOnly-4x16-ResNet50 | 73.8 / 90.9 | 74.5 / 91.4 | 75.2 / 91.6 | 75.2 / 91.7 | 74.5 / 91.1 | 76.6 / 92.5 |
+
+### Citing OmniSource
+
+If you find OmniSource useful for your research, please consider citing the paper using the following BibTeX entry.
+
+```
+@article{duan2020omni,
+ title={Omni-sourced Webly-supervised Learning for Video Recognition},
+ author={Duan, Haodong and Zhao, Yue and Xiong, Yuanjun and Liu, Wentao and Lin, Dahua},
+ journal={arXiv preprint arXiv:2003.13042},
+ year={2020}
+}
+```
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/pipeline.png b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/pipeline.png
new file mode 100644
index 0000000..a3e3a2a
Binary files /dev/null and b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/pipeline.png differ
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_googleimage_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_googleimage_rgb.py
new file mode 100644
index 0000000..2dd3002
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_googleimage_rgb.py
@@ -0,0 +1,149 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=200,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+gg_root = 'data/OmniSource/googleimage_200'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_gg = ('data/OmniSource/annotations/googleimage_200/'
+ 'tsn_8seg_googleimage_200_wodup.txt')
+
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_gg_pipeline = [
+ dict(type='ImageDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='BuildPseudoClip', clip_len=8),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=2,
+ train=[
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ dict(
+ type='ImageDataset',
+ ann_file=ann_file_gg,
+ data_prefix=gg_root,
+ pipeline=train_gg_pipeline)
+ ],
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.15, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=8)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=8, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/omnisource/'
+ 'slowonly_r50_8x8x1_256e_minikinetics_googleimage_rgb')
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_insvideo_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_insvideo_rgb.py
new file mode 100644
index 0000000..6725002
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_insvideo_rgb.py
@@ -0,0 +1,153 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=200,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+iv_root = 'data/OmniSource/insvideo_200'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_iv = ('data/OmniSource/annotations/insvideo_200/'
+ 'slowonly_8x8_insvideo_200_wodup.txt')
+
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_iv_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=2,
+ train=[
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_iv,
+ data_prefix=iv_root,
+ pipeline=train_iv_pipeline,
+ num_classes=200,
+ sample_by_class=True,
+ power=0.5)
+ ],
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.15, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=8)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=8, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/omnisource/'
+ 'slowonly_r50_8x8x1_256e_minikinetics_insvideo_rgb')
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_kineticsraw_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_kineticsraw_rgb.py
new file mode 100644
index 0000000..f0ccd57
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_kineticsraw_rgb.py
@@ -0,0 +1,152 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=200,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+kraw_root = 'data/OmniSource/kinetics_raw_200_train'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_kraw = ('data/OmniSource/annotations/kinetics_raw_200/'
+ 'slowonly_8x8_kinetics_raw_200.json')
+
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_kraw_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=2,
+ train=[
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ dict(
+ type='RawVideoDataset',
+ ann_file=ann_file_kraw,
+ data_prefix=kraw_root,
+ pipeline=train_kraw_pipeline,
+ clipname_tmpl='part_{}.mp4',
+ sampling_strategy='positive')
+ ],
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.15, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=8)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=8, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/omnisource/'
+ 'slowonly_r50_8x8x1_256e_minikinetics_kineticsraw_rgb')
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_omnisource_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_omnisource_rgb.py
new file mode 100644
index 0000000..9aeed38
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_omnisource_rgb.py
@@ -0,0 +1,200 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=200,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+web_root = 'data/OmniSource/'
+iv_root = 'data/OmniSource/insvideo_200'
+kraw_root = 'data/OmniSource/kinetics_raw_200_train'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_web = ('data/OmniSource/annotations/webimage_200/'
+ 'tsn_8seg_webimage_200_wodup.txt')
+ann_file_iv = ('data/OmniSource/annotations/insvideo_200/'
+ 'slowonly_8x8_insvideo_200_wodup.txt')
+ann_file_kraw = ('data/OmniSource/annotations/kinetics_raw_200/'
+ 'slowonly_8x8_kinetics_raw_200.json')
+
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_web_pipeline = [
+ dict(type='ImageDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='BuildPseudoClip', clip_len=8),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_iv_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_kraw_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=1,
+ train_ratio=[2, 1, 1, 1],
+ train=[
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ dict(
+ type='ImageDataset',
+ ann_file=ann_file_web,
+ data_prefix=web_root,
+ pipeline=train_web_pipeline,
+ num_classes=200,
+ sample_by_class=True,
+ power=0.5),
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_iv,
+ data_prefix=iv_root,
+ pipeline=train_iv_pipeline,
+ num_classes=200,
+ sample_by_class=True,
+ power=0.5),
+ dict(
+ type='RawVideoDataset',
+ ann_file=ann_file_kraw,
+ data_prefix=kraw_root,
+ pipeline=train_kraw_pipeline,
+ clipname_tmpl='part_{}.mp4',
+ sampling_strategy='positive')
+ ],
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.15, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=8)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=8, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/omnisource/'
+ 'slowonly_r50_8x8x1_256e_minikinetics_omnisource_rgb')
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_rgb.py
new file mode 100644
index 0000000..5b12fac
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_rgb.py
@@ -0,0 +1,125 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=200,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.15, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=8)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=8, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/omnisource/slowonly_r50_8x8x1_256e_minikinetics_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_webimage_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_webimage_rgb.py
new file mode 100644
index 0000000..62fafcf
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/slowonly_r50_8x8x1_256e_minikinetics/slowonly_r50_8x8x1_256e_minikinetics_webimage_rgb.py
@@ -0,0 +1,152 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=200,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+web_root = 'data/OmniSource/'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_web = ('data/OmniSource/annotations/webimage_200/'
+ 'tsn_8seg_webimage_200_wodup.txt')
+
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_web_pipeline = [
+ dict(type='ImageDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='BuildPseudoClip', clip_len=8),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=2,
+ train=[
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ dict(
+ type='ImageDataset',
+ ann_file=ann_file_web,
+ data_prefix=web_root,
+ pipeline=train_web_pipeline,
+ num_classes=200,
+ sample_by_class=True,
+ power=0.5)
+ ],
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.15, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=8)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=8, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/omnisource/'
+ 'slowonly_r50_8x8x1_256e_minikinetics_webimage_rgb')
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_googleimage_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_googleimage_rgb.py
new file mode 100644
index 0000000..3599651
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_googleimage_rgb.py
@@ -0,0 +1,146 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=200,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+omnisource = True
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+gg_root = 'data/OmniSource/googleimage_200'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_gg = ('data/OmniSource/annotations/googleimage_200/'
+ 'tsn_8seg_googleimage_200_wodup.txt')
+
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_gg_pipeline = [
+ dict(type='ImageDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ omni_videos_per_gpu=[12, 64],
+ workers_per_gpu=2,
+ train=[
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ dict(
+ type='ImageDataset',
+ ann_file=ann_file_gg,
+ data_prefix=gg_root,
+ pipeline=train_gg_pipeline)
+ ],
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.00375, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/omnisource/'
+ 'tsn_r50_1x1x8_100e_minikinetics_googleimage_rgb')
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_insvideo_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_insvideo_rgb.py
new file mode 100644
index 0000000..9a51b51
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_insvideo_rgb.py
@@ -0,0 +1,150 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=200,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+omnisource = True
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+iv_root = 'data/OmniSource/insvideo_200'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_iv = ('data/OmniSource/annotations/insvideo_200/'
+ 'slowonly_8x8_insvideo_200_wodup.txt')
+
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_iv_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=2,
+ train=[
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_iv,
+ data_prefix=iv_root,
+ pipeline=train_iv_pipeline,
+ num_classes=200,
+ sample_by_class=True,
+ power=0.5)
+ ],
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.00375, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/omnisource/'
+ 'tsn_r50_1x1x8_100e_minikinetics_insvideo_rgb')
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_kineticsraw_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_kineticsraw_rgb.py
new file mode 100644
index 0000000..a70b4ff
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_kineticsraw_rgb.py
@@ -0,0 +1,149 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=200,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+omnisource = True
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+kraw_root = 'data/OmniSource/kinetics_raw_200_train'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_kraw = ('data/OmniSource/annotations/kinetics_raw_200/'
+ 'slowonly_8x8_kinetics_raw_200.json')
+
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_kraw_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=2,
+ train=[
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ dict(
+ type='RawVideoDataset',
+ ann_file=ann_file_kraw,
+ data_prefix=kraw_root,
+ pipeline=train_kraw_pipeline,
+ clipname_tmpl='part_{}.mp4',
+ sampling_strategy='positive')
+ ],
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.00375, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/omnisource/'
+ 'tsn_r50_1x1x8_100e_minikinetics_kineticsraw_rgb')
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_omnisource_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_omnisource_rgb.py
new file mode 100644
index 0000000..f084944
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_omnisource_rgb.py
@@ -0,0 +1,197 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=200,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+omnisource = True
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+web_root = 'data/OmniSource/'
+iv_root = 'data/OmniSource/insvideo_200'
+kraw_root = 'data/OmniSource/kinetics_raw_200_train'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_web = ('data/OmniSource/annotations/webimage_200/'
+ 'tsn_8seg_webimage_200_wodup.txt')
+ann_file_iv = ('data/OmniSource/annotations/insvideo_200/'
+ 'slowonly_8x8_insvideo_200_wodup.txt')
+ann_file_kraw = ('data/OmniSource/annotations/kinetics_raw_200/'
+ 'slowonly_8x8_kinetics_raw_200.json')
+
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_web_pipeline = [
+ dict(type='ImageDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_iv_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_kraw_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ omni_videos_per_gpu=[12, 64, 12, 12],
+ train_ratio=[2, 1, 1, 1],
+ workers_per_gpu=1,
+ train=[
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ dict(
+ type='ImageDataset',
+ ann_file=ann_file_web,
+ data_prefix=web_root,
+ pipeline=train_web_pipeline,
+ num_classes=200,
+ sample_by_class=True,
+ power=0.5),
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_iv,
+ data_prefix=iv_root,
+ pipeline=train_iv_pipeline,
+ num_classes=200,
+ sample_by_class=True,
+ power=0.5),
+ dict(
+ type='RawVideoDataset',
+ ann_file=ann_file_kraw,
+ data_prefix=kraw_root,
+ pipeline=train_kraw_pipeline,
+ clipname_tmpl='part_{}.mp4',
+ sampling_strategy='positive')
+ ],
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.00375, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/omnisource/'
+ 'tsn_r50_1x1x8_100e_minikinetics_omnisource_rgb')
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_rgb.py
new file mode 100644
index 0000000..00d390c
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_rgb.py
@@ -0,0 +1,120 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=200,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.00375, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/omnisource/tsn_r50_1x1x8_100e_minikinetics_rgb'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_webimage_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_webimage_rgb.py
new file mode 100644
index 0000000..4502bdc
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/omnisource/tsn_r50_1x1x8_100e_minikinetics/tsn_r50_1x1x8_100e_minikinetics_webimage_rgb.py
@@ -0,0 +1,149 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=200,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+omnisource = True
+# dataset settings
+dataset_type = 'VideoDataset'
+# The flag indicates using joint training
+omnisource = True
+
+data_root = 'data/OmniSource/kinetics_200_train'
+data_root_val = 'data/OmniSource/kinetics_200_val'
+web_root = 'data/OmniSource/'
+
+ann_file_train = 'data/OmniSource/annotations/kinetics_200/k200_train.txt'
+ann_file_web = ('data/OmniSource/annotations/webimage_200/'
+ 'tsn_8seg_webimage_200_wodup.txt')
+
+ann_file_val = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+ann_file_test = 'data/OmniSource/annotations/kinetics_200/k200_val.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+train_web_pipeline = [
+ dict(type='ImageDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=12,
+ omni_videos_per_gpu=[12, 64],
+ workers_per_gpu=2,
+ train=[
+ dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ dict(
+ type='ImageDataset',
+ ann_file=ann_file_web,
+ data_prefix=web_root,
+ pipeline=train_web_pipeline,
+ num_classes=200,
+ sample_by_class=True,
+ power=0.5)
+ ],
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.00375, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/omnisource/'
+ 'tsn_r50_1x1x8_100e_minikinetics_webimage_rgb')
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/README.md
new file mode 100644
index 0000000..51d64cb
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/README.md
@@ -0,0 +1,62 @@
+# R2plus1D
+
+## Introduction
+```
+@inproceedings{tran2018closer,
+ title={A closer look at spatiotemporal convolutions for action recognition},
+ author={Tran, Du and Wang, Heng and Torresani, Lorenzo and Ray, Jamie and LeCun, Yann and Paluri, Manohar},
+ booktitle={Proceedings of the IEEE conference on Computer Vision and Pattern Recognition},
+ pages={6450--6459},
+ year={2018}
+}
+```
+
+## Model Zoo
+
+### Kinetics-400
+
+|config | resolution | gpus | backbone | pretrain| top1 acc| top5 acc | inference_time(video/s) | gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[r2plus1d_r34_8x8x1_180e_kinetics400_rgb](/configs/recognition/r2plus1d/r2plus1d_r34_8x8x1_180e_kinetics400_rgb.py) | short-side 256|8x4| ResNet34|None |67.30|87.65|x|5019|[ckpt](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_256p_8x8x1_180e_kinetics400_rgb/r2plus1d_r34_256p_8x8x1_180e_kinetics400_rgb_20200729-aa94765e.pth)|[log](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_256p_8x8x1_180e_kinetics400_rgb/20200728_021421.log)|[json](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_256p_8x8x1_180e_kinetics400_rgb/20200728_021421.log.json)|
+|[r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb](/configs/recognition/r2plus1d/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb.py) | short-side 256|8| ResNet34|None |67.3|87.8|x|5019|[ckpt](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb_20200826-ab35a529.pth)|[log](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb/20200724_201360.log.json)|[json](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb/20200724_201360.log)|
+|[r2plus1d_r34_8x8x1_180e_kinetics400_rgb](/configs/recognition/r2plus1d/r2plus1d_r34_8x8x1_180e_kinetics400_rgb.py) | short-side 320|8x2| ResNet34|None |68.68|88.36|1.6 (80x3 frames)|5019|[ckpt](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_8x8x1_180e_kinetics400_rgb/r2plus1d_r34_8x8x1_180e_kinetics400_rgb_20200618-3fce5629.pth)| [log](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_8x8x1_180e_kinetics400_rgb/r21d_8x8.log)| [json](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_8x8x1_180e_kinetics400_rgb/r2plus1d_r34_8x8_69.58_88.36.log.json)|
+|[r2plus1d_r34_32x2x1_180e_kinetics400_rgb](/configs/recognition/r2plus1d/r2plus1d_r34_32x2x1_180e_kinetics400_rgb.py) |short-side 320|8x2| ResNet34|None |74.60|91.59|0.5 (320x3 frames)|12975| [ckpt](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_32x2x1_180e_kinetics400_rgb/r2plus1d_r34_32x2x1_180e_kinetics400_rgb_20200618-63462eb3.pth) | [log](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_32x2x1_180e_kinetics400_rgb/r21d_32x2.log)| [json](https://download.openmmlab.com/mmaction/recognition/r2plus1d/r2plus1d_r34_32x2x1_180e_kinetics400_rgb/r2plus1d_r34_32x2_74.6_91.6.log.json)|
+
+Notes:
+1. The **gpus** indicates the number of gpu we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+
+For more details on data preparation, you can refer to Kinetics400 in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train R(2+1)D model on Kinetics-400 dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/recognition/r2plus1d/r2plus1d_r34_8x8x1_180e_kinetics400_rgb.py \
+ --work-dir work_dirs/r2plus1d_r34_3d_8x8x1_180e_kinetics400_rgb \
+ --validate --seed 0 --deterministic
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test R(2+1)D model on Kinetics-400 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/r2plus1d/r2plus1d_r34_8x8x1_180e_kinetics400_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json --average-clips=prob
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_32x2x1_180e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_32x2x1_180e_kinetics400_rgb.py
new file mode 100644
index 0000000..0ef54e5
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_32x2x1_180e_kinetics400_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet2Plus1d',
+ depth=34,
+ pretrained=None,
+ pretrained2d=False,
+ norm_eval=False,
+ conv_cfg=dict(type='Conv2plus1d'),
+ norm_cfg=dict(type='SyncBN', requires_grad=True, eps=1e-3),
+ conv1_kernel=(3, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(1, 1, 1, 1),
+ spatial_strides=(1, 2, 2, 2),
+ temporal_strides=(1, 2, 2, 2),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=512,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=6,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ test_mode=True),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ test_mode=True))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.075, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 180
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/r2plus1d_r34_3d_32x2x1_180e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_8x8x1_180e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_8x8x1_180e_kinetics400_rgb.py
new file mode 100644
index 0000000..8b540d0
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_8x8x1_180e_kinetics400_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet2Plus1d',
+ depth=34,
+ pretrained=None,
+ pretrained2d=False,
+ norm_eval=False,
+ conv_cfg=dict(type='Conv2plus1d'),
+ norm_cfg=dict(type='SyncBN', requires_grad=True, eps=1e-3),
+ conv1_kernel=(3, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(1, 1, 1, 1),
+ spatial_strides=(1, 2, 2, 2),
+ temporal_strides=(1, 2, 2, 2),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=512,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ test_mode=True),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ test_mode=True))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.1, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 180
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/r2plus1d_r34_8x8x1_180e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb.py
new file mode 100644
index 0000000..8282b94
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb.py
@@ -0,0 +1,130 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet2Plus1d',
+ depth=34,
+ pretrained=None,
+ pretrained2d=False,
+ norm_eval=False,
+ conv_cfg=dict(type='Conv2plus1d'),
+ norm_cfg=dict(type='SyncBN', requires_grad=True, eps=1e-3),
+ act_cfg=dict(type='ReLU'),
+ conv1_kernel=(3, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(1, 1, 1, 1),
+ spatial_strides=(1, 2, 2, 2),
+ temporal_strides=(1, 2, 2, 2),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=512,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics400/videos_train'
+data_root_val = 'data/kinetics400/videos_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_videos.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_videos.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ test_mode=True),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ test_mode=True))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.2, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 180
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/r2plus1d_r34_video_3d_8x8x1_180e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_video_inference_8x8x1_180e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_video_inference_8x8x1_180e_kinetics400_rgb.py
new file mode 100644
index 0000000..daf0434
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/r2plus1d/r2plus1d_r34_video_inference_8x8x1_180e_kinetics400_rgb.py
@@ -0,0 +1,56 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet2Plus1d',
+ depth=34,
+ pretrained=None,
+ pretrained2d=False,
+ norm_eval=False,
+ conv_cfg=dict(type='Conv2plus1d'),
+ norm_cfg=dict(type='SyncBN', requires_grad=True, eps=1e-3),
+ act_cfg=dict(type='ReLU'),
+ conv1_kernel=(3, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(1, 1, 1, 1),
+ spatial_strides=(1, 2, 2, 2),
+ temporal_strides=(1, 2, 2, 2),
+ zero_init_residual=False),
+ cls_head=dict(
+ type='I3DHead',
+ num_classes=400,
+ in_channels=512,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/README.md
new file mode 100644
index 0000000..6e38fd6
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/README.md
@@ -0,0 +1,63 @@
+# SlowFast
+
+## Introduction
+```
+@inproceedings{feichtenhofer2019slowfast,
+ title={Slowfast networks for video recognition},
+ author={Feichtenhofer, Christoph and Fan, Haoqi and Malik, Jitendra and He, Kaiming},
+ booktitle={Proceedings of the IEEE international conference on computer vision},
+ pages={6202--6211},
+ year={2019}
+}
+```
+
+## Model Zoo
+
+### Kinetics-400
+
+|config | resolution | gpus | backbone |pretrain| top1 acc| top5 acc | inference_time(video/s) | gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[slowfast_r50_4x16x1_256e_kinetics400_rgb](/configs/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb.py) |short-side 256|8x4| ResNet50|None |74.75|91.73|x|6203|[ckpt](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_256p_4x16x1_256e_kinetics400_rgb/slowfast_r50_256p_4x16x1_256e_kinetics400_rgb_20200728-145f1097.pth)|[log](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_256p_4x16x1_256e_kinetics400_rgb/20200731_151706.log)|[json](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_256p_4x16x1_256e_kinetics400_rgb/20200731_151706.log.json)|
+|[slowfast_r50_video_4x16x1_256e_kinetics400_rgb](/configs/recognition/slowfast/slowfast_r50_video_4x16x1_256e_kinetics400_rgb.py) |short-side 256|8| ResNet50|None |74.34|91.58|x|6203|[ckpt](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_video_4x16x1_256e_kinetics400_rgb/slowfast_r50_video_4x16x1_256e_kinetics400_rgb_20200826-f85b90c5.pth)|[log](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_video_4x16x1_256e_kinetics400_rgb/20200812_160237.log)|[json](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_video_4x16x1_256e_kinetics400_rgb/20200812_160237.log.json)|
+|[slowfast_r50_4x16x1_256e_kinetics400_rgb](/configs/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb.py) |short-side 320|8x3| ResNet50|None |75.64|92.3|1.6 ((32+4)x10x3 frames)|6203|[ckpt](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb/slowfast_r50_4x16x1_256e_kinetics400_rgb_20200704-bcde7ed7.pth)| [log](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb/20200704_232901.log)| [json](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb/20200704_232901.log.json)|
+|[slowfast_r50_8x8x1_256e_kinetics400_rgb](/configs/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb.py) |short-side 256|8x4| ResNet50 |None |75.61|92.34|x|9062|[ckpt](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_256p_8x8x1_256e_kinetics400_rgb/slowfast_r50_256p_8x8x1_256e_kinetics400_rgb_20200810-863812c2.pth)|[log](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_256p_8x8x1_256e_kinetics400_rgb/20200731_151537.log)|[json](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_256p_8x8x1_256e_kinetics400_rgb/20200731_151537.log.json)|
+|[slowfast_r50_8x8x1_256e_kinetics400_rgb](/configs/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb.py) |short-side 320|8x3| ResNet50 |None|76.94|92.8|1.3 ((32+8)x10x3 frames)|9062| [ckpt](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb/slowfast_r50_8x8x1_256e_kinetics400_rgb_20200716-73547d2b.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb/20200716_192653.log)| [json](https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb/20200716_192653.log.json)|
+
+Notes:
+1. The **gpus** indicates the number of gpu we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+
+For more details on data preparation, you can refer to Kinetics400 in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train SlowFast model on Kinetics-400 dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb.py \
+ --work-dir work_dirs/slowfast_r50_4x16x1_256e_kinetics400_rgb \
+ --validate --seed 0 --deterministic
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test SlowFast model on Kinetics-400 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json --average-clips=prob
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_bnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_bnn.py
new file mode 100644
index 0000000..3209b61
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_bnn.py
@@ -0,0 +1,143 @@
+model = dict(
+ type='Recognizer3DBNN',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=4, # tau
+ speed_ratio=4, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ fusion_kernel=7,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastBNNHead',
+ in_channels=2304, # 2048+256
+ num_classes=101,
+ spatial_type='avg',
+ dropout_ratio=0))
+train_cfg = dict(loss_weight=1e-6, npass=2)
+test_cfg = dict(average_clips='prob', npass=10)
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9,
+ weight_decay=0.0001, nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr=0,
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=5)
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/finetune_ucf101_slowfast_bnn'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb/slowfast_r50_8x8x1_256e_kinetics400_rgb_20200716-73547d2b.pth'
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_dnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_dnn.py
new file mode 100644
index 0000000..e11baab
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_dnn.py
@@ -0,0 +1,143 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=4, # tau
+ speed_ratio=4, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ fusion_kernel=7,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastHead',
+ in_channels=2304, # 2048+256
+ num_classes=101,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9,
+ weight_decay=0.0001, nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr=0,
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=5)
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/finetune_ucf101_slowfast_dnn'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb/slowfast_r50_8x8x1_256e_kinetics400_rgb_20200716-73547d2b.pth'
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_edlnokl_avuc_debias.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_edlnokl_avuc_debias.py
new file mode 100644
index 0000000..18e1afa
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_edlnokl_avuc_debias.py
@@ -0,0 +1,161 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=4, # tau
+ speed_ratio=4, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ fusion_kernel=7,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastHead',
+ loss_cls=evidence_loss,
+ in_channels=2304, # 2048+256
+ num_classes=101,
+ spatial_type='avg',
+ dropout_ratio=0.5),
+ debias_head=dict(
+ type='DebiasHead',
+ loss_cls=evidence_loss, # actually not used!
+ loss_factor=0.1,
+ num_classes=101,
+ in_channels=2048, # only slow features are debiased
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9,
+ weight_decay=0.0001, nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr=0,
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=5)
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/finetune_ucf101_slowfast_dnn'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb/slowfast_r50_8x8x1_256e_kinetics400_rgb_20200716-73547d2b.pth'
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_rpl.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_rpl.py
new file mode 100644
index 0000000..cbbdedd
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/finetune_ucf101_slowfast_rpl.py
@@ -0,0 +1,146 @@
+model = dict(
+ type='Recognizer3DRPL',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=4, # tau
+ speed_ratio=4, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ fusion_kernel=7,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastRPLHead',
+ loss_cls=dict(type='RPLoss',
+ temperature=1,
+ weight_pl=0.1),
+ in_channels=2304, # 2048+256
+ num_classes=101,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9,
+ weight_decay=0.0001, nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr=0,
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=5)
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/finetune_ucf101_slowfast_rpl'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb/slowfast_r50_8x8x1_256e_kinetics400_rgb_20200716-73547d2b.pth'
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_bnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_bnn.py
new file mode 100644
index 0000000..8d2f181
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_bnn.py
@@ -0,0 +1,66 @@
+model = dict(
+ type='Recognizer3DBNN',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=4, # tau
+ speed_ratio=4, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ fusion_kernel=7,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastBNNHead',
+ in_channels=2304, # 2048+256
+ num_classes=101,
+ spatial_type='avg',
+ dropout_ratio=0))
+test_cfg = dict(average_clips='prob', npass=10)
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=2,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ start_index=0,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_dnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_dnn.py
new file mode 100644
index 0000000..028841a
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_dnn.py
@@ -0,0 +1,66 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=4, # tau
+ speed_ratio=4, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ fusion_kernel=7,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastHead',
+ in_channels=2304, # 2048+256
+ num_classes=101,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=2,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ start_index=0,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_enn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_enn.py
new file mode 100644
index 0000000..3c12f7e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_enn.py
@@ -0,0 +1,75 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=4, # tau
+ speed_ratio=4, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ fusion_kernel=7,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastHead',
+ loss_cls=dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ annealing_method='exp'),
+ in_channels=2304, # 2048+256
+ num_classes=101,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+evidence='exp' # only used for EDL
+test_cfg = dict(average_clips='score')
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ # dict(type='OpenCVInit', num_threads=1),
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ # dict(type='OpenCVDecode'),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ start_index=0,
+ pipeline=test_pipeline))
+dist_params = dict(backend='nccl')
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_rpl.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_rpl.py
new file mode 100644
index 0000000..414dc8e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/inference_slowfast_rpl.py
@@ -0,0 +1,69 @@
+model = dict(
+ type='Recognizer3DRPL',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=4, # tau
+ speed_ratio=4, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ fusion_kernel=7,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastRPLHead',
+ loss_cls=dict(type='RPLoss',
+ temperature=1,
+ weight_pl=0.1),
+ in_channels=2304, # 2048+256
+ num_classes=101,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=2,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ start_index=0,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..1adbf2c
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb.py
@@ -0,0 +1,136 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=8, # tau
+ speed_ratio=8, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastHead',
+ in_channels=2304, # 2048+256
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.1, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr=0,
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=34)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowfast_r50_3d_4x16x1_256e_kinetics400_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..94486b7
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb.py
@@ -0,0 +1,137 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=4, # tau
+ speed_ratio=4, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ fusion_kernel=7,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastHead',
+ in_channels=2304, # 2048+256
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.1, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr=0,
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=34)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowfast_r50_3d_8x8x1_256e_kinetics400_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_video_4x16x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_video_4x16x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..4dba180
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_video_4x16x1_256e_kinetics400_rgb.py
@@ -0,0 +1,139 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=8, # tau
+ speed_ratio=8, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastHead',
+ in_channels=2304, # 2048+256
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics400/videos_train'
+data_root_val = 'data/kinetics400/videos_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_videos.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_videos.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.1, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr=0,
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=34)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowfast_r50_video_3d_4x16x1_256e_kinetics400_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_video_inference_4x16x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_video_inference_4x16x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..a1f1d03
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowfast/slowfast_r50_video_inference_4x16x1_256e_kinetics400_rgb.py
@@ -0,0 +1,68 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowFast',
+ pretrained=None,
+ resample_rate=8, # tau
+ speed_ratio=8, # alpha
+ channel_ratio=8, # beta_inv
+ slow_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=True,
+ conv1_kernel=(1, 7, 7),
+ dilations=(1, 1, 1, 1),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ fast_pathway=dict(
+ type='resnet3d',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ base_channels=8,
+ conv1_kernel=(5, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ norm_eval=False)),
+ cls_head=dict(
+ type='SlowFastHead',
+ in_channels=2304, # 2048+256
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+
+# model training and testing settings
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/README.md
new file mode 100644
index 0000000..a0c48ac
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/README.md
@@ -0,0 +1,106 @@
+# SlowOnly
+
+## Introduction
+```
+@inproceedings{feichtenhofer2019slowfast,
+ title={Slowfast networks for video recognition},
+ author={Feichtenhofer, Christoph and Fan, Haoqi and Malik, Jitendra and He, Kaiming},
+ booktitle={Proceedings of the IEEE international conference on computer vision},
+ pages={6202--6211},
+ year={2019}
+}
+```
+
+## Model Zoo
+
+### Kinetics-400
+
+|config | resolution | gpus | backbone |pretrain| top1 acc| top5 acc | inference_time(video/s) | gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[slowonly_r50_4x16x1_256e_kinetics400_rgb](/configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb.py)|short-side 256|8x4| ResNet50 | None |72.76|90.51|x|3168|[ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_256p_4x16x1_256e_kinetics400_rgb/slowonly_r50_256p_4x16x1_256e_kinetics400_rgb_20200820-bea7701f.pth)|[log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_256p_4x16x1_256e_kinetics400_rgb/20200817_001411.log)|[json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_256p_4x16x1_256e_kinetics400_rgb/20200817_001411.log.json)|
+|[slowonly_r50_video_4x16x1_256e_kinetics400_rgb](/configs/recognition/slowonly/slowonly_r50_video_4x16x1_256e_kinetics400_rgb.py)|short-side 320|8x2| ResNet50 | None |72.90|90.82|x|8472|[ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_video_320p_4x16x1_256e_kinetics400_rgb/slowonly_r50_video_320p_4x16x1_256e_kinetics400_rgb_20201014-c9cdc656.pth)|[log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_video_320p_4x16x1_256e_kinetics400_rgb/slowonly_r50_video_320p_4x16x1_256e_kinetics400_rgb_20201014.log)|[json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_video_320p_4x16x1_256e_kinetics400_rgb/slowonly_r50_video_320p_4x16x1_256e_kinetics400_rgb_20201014.json)|
+|[slowonly_r50_8x8x1_256e_kinetics400_rgb](/configs/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_rgb.py) |short-side 256|8x4| ResNet50 | None |74.42|91.49|x|5820|[ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_256p_8x8x1_256e_kinetics400_rgb/slowonly_r50_256p_8x8x1_256e_kinetics400_rgb_20200820-75851a7d.pth)|[log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_256p_8x8x1_256e_kinetics400_rgb/20200817_003320.log)|[json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_256p_8x8x1_256e_kinetics400_rgb/20200817_003320.log.json)|
+|[slowonly_r50_4x16x1_256e_kinetics400_rgb](/configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb.py)|short-side 320|8x2| ResNet50 | None |73.02|90.77|4.0 (40x3 frames)|3168|[ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb/slowonly_r50_4x16x1_256e_kinetics400_rgb_20200704-a69556c6.pth)| [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb/so_4x16.log)| [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb/slowonly_r50_4x16_73.02_90.77.log.json)|
+|[slowonly_r50_8x8x1_256e_kinetics400_rgb](/configs/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_rgb.py) |short-side 320|8x3| ResNet50 | None |74.93|91.92|2.3 (80x3 frames)|5820| [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_rgb/slowonly_r50_8x8x1_256e_kinetics400_rgb_20200703-a79c555a.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_rgb/so_8x8.log)| [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_rgb/slowonly_r50_8x8_74.93_91.92.log.json)|
+|[slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb](/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb.py)|short-side 320|8x2| ResNet50 | ImageNet |73.39|91.12|x|3168|[ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb/slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb_20200912-1e8fc736.pth)|[log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb/slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb_20200912.log)|[json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb/slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb_20200912.json)|
+|[slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb](/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb.py) |short-side 320|8x4| ResNet50 | ImageNet |75.55|92.04|x|5820|[ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb/slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb_20200912-3f9ce182.pth)|[log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb/slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb_20200912.log)|[json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb/slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb_20200912.json)|
+|[slowonly_r50_4x16x1_256e_kinetics400_flow](/configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_flow.py)|short-side 320|8x2| ResNet50 | ImageNet |61.79|83.62|x|8450| [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_flow/slowonly_r50_4x16x1_256e_kinetics400_flow_20200704-decb8568.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_flow/slowonly_r50_4x16x1_256e_kinetics400_flow_61.8_83.6.log) | [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_flow/slowonly_r50_4x16x1_256e_kinetics400_flow_61.8_83.6.log.json)|
+|[slowonly_r50_8x8x1_196e_kinetics400_flow](/configs/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_flow.py) |short-side 320|8x4| ResNet50 | ImageNet |65.76|86.25|x|8455| [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_flow/slowonly_r50_8x8x1_256e_kinetics400_flow_20200704-6b384243.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_flow/slowonly_r50_8x8x1_196e_kinetics400_flow_65.8_86.3.log) | [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_flow/slowonly_r50_8x8x1_196e_kinetics400_flow_65.8_86.3.log.json)|
+
+### Kinetics-400 Data Benchmark
+In data benchmark, we compare two different data preprocessing methods: (1) Resize video to 340x256, (2) Resize the short edge of video to 320px, (3) Resize the short edge of video to 256px.
+
+| config | resolution | gpus | backbone | Input | pretrain | top1 acc | top5 acc | testing protocol | ckpt | log | json |
+| :----------------------------------------------------------- | :------------: | :--: | :------: | :---: | :------: | :------: | :------: | :----------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb](/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb.py) | 340x256 | 8x2 | ResNet50 | 4x16 | None | 71.61 | 90.05 | 10 clips x 3 crops | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb/slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb_20200803-dadca1a3.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb/slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb_20200803.log) | [json](https://download.openmmlab.com/mmaction/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb/slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb_20200803.json) |
+| [slowonly_r50_randomresizedcrop_320p_4x16x1_256e_kinetics400_rgb](/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_320p_4x16x1_256e_kinetics400_rgb.py) | short-side 320 | 8x2 | ResNet50 | 4x16 | None | 73.02 | 90.77 | 10 clips x 3 crops | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb/slowonly_r50_4x16x1_256e_kinetics400_rgb_20200704-a69556c6.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb/so_4x16.log) | [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb/slowonly_r50_4x16_73.02_90.77.log.json) |
+| [slowonly_r50_randomresizedcrop_256p_4x16x1_256e_kinetics400_rgb](/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_256p_4x16x1_256e_kinetics400_rgb.py) | short-side 256 | 8x4 | ResNet50 | 4x16 | None | 72.76 | 90.51 | 10 clips x 3 crops | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_256p_4x16x1_256e_kinetics400_rgb/slowonly_r50_256p_4x16x1_256e_kinetics400_rgb_20200820-bea7701f.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_256p_4x16x1_256e_kinetics400_rgb/20200817_001411.log) | [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_256p_4x16x1_256e_kinetics400_rgb/20200817_001411.log.json) |
+
+### Kinetics-400 OmniSource Experiments
+
+| config | resolution | backbone | pretrain | w. OmniSource | top1 acc | top5 acc | ckpt | log | json |
+| :----------------------------------------------------------: | :------------: | :-------: | :------: | :----------------: | :------: | :------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [slowonly_r50_4x16x1_256e_kinetics400_rgb](/configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb.py) | short-side 320 | ResNet50 | None | :x: | 73.0 | 90.8 | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb/slowonly_r50_4x16x1_256e_kinetics400_rgb_20200704-a69556c6.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb/so_4x16.log) | [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb/slowonly_r50_4x16_73.02_90.77.log.json) |
+| x | x | ResNet50 | None | :heavy_check_mark: | 76.8 | 92.5 | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/omni/slowonly_r50_omni_4x16x1_kinetics400_rgb_20200926-51b1f7ea.pth) | x | x |
+| [slowonly_r101_8x8x1_196e_kinetics400_rgb](/configs/recognition/slowonly/slowonly_r101_8x8x1_196e_kinetics400_rgb.py) | x | ResNet101 | None | :x: | 76.5 | 92.7 | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/omni/slowonly_r101_without_omni_8x8x1_kinetics400_rgb_20200926-0c730aef.pth) | x | x |
+| x | x | ResNet101 | None | :heavy_check_mark: | 80.4 | 94.4 | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/omni/slowonly_r101_omni_8x8x1_kinetics400_rgb_20200926-b5dbb701.pth) | x | x |
+
+### Kinetics-600
+
+| config | resolution | gpus | backbone | pretrain | top1 acc | top5 acc | ckpt | log | json |
+| :----------------------------------------------------------- | :------------: | :--: | :------: | :------: | :------: | :------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [slowonly_r50_video_8x8x1_256e_kinetics600_rgb](/configs/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics600_rgb.py) | short-side 256 | 8x4 | ResNet50 | None | 77.5 | 93.7 | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics600_rgb/slowonly_r50_video_8x8x1_256e_kinetics600_rgb_20201015-81e5153e.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics600_rgb/slowonly_r50_video_8x8x1_256e_kinetics600_rgb_20201015.log) | [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics600_rgb/slowonly_r50_video_8x8x1_256e_kinetics600_rgb_20201015.json) |
+
+### Kinetics-700
+
+| config | resolution | gpus | backbone | pretrain | top1 acc | top5 acc | ckpt | log | json |
+| :----------------------------------------------------------- | :------------: | :--: | :------: | :------: | :------: | :------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [slowonly_r50_video_8x8x1_256e_kinetics700_rgb](/configs/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics700_rgb.py) | short-side 256 | 8x4 | ResNet50 | None | 65.0 | 86.1 | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics700_rgb/slowonly_r50_video_8x8x1_256e_kinetics700_rgb_20201015-9250f662.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics700_rgb/slowonly_r50_video_8x8x1_256e_kinetics700_rgb_20201015.log) | [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics700_rgb/slowonly_r50_video_8x8x1_256e_kinetics700_rgb_20201015.json) |
+
+### GYM99
+
+| config | resolution | gpus | backbone | pretrain | top1 acc | mean class acc | ckpt | log | json |
+| :----------------------------------------------------------- | :------------: | :--: | :------: | :------: | :------: | :------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb](/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb.py) | short-side 256 | 8x2 | ResNet50 | ImageNet | 79.3 | 70.2 | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb_20201111-a9c34b54.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb_20201111.log) | [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb_20201111.json) |
+| [slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow](/configs/recognition/slowonly/slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow.py) | short-side 256 | 8x2 | ResNet50 | Kinetics | 80.3 | 71.0 | [ckpt](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow/slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow_20201111-66ecdb3c.pth) | [log](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow/slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow_20201111.log) | [json](https://download.openmmlab.com/mmaction/recognition/slowonly/slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow/slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow_20201111.json) |
+| 1: 1 Fusion | | | | | 83.7 | 74.8 | | | |
+
+Notes:
+
+1. The **gpus** indicates the number of gpu we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+
+For more details on data preparation, you can refer to Kinetics400 in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train SlowOnly model on Kinetics-400 dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb.py \
+ --work-dir work_dirs/slowonly_r50_4x16x1_256e_kinetics400_rgb \
+ --validate --seed 0 --deterministic
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test SlowOnly model on Kinetics-400 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json --average-clips=prob
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_256p_4x16x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_256p_4x16x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..f4564bd
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_256p_4x16x1_256e_kinetics400_rgb.py
@@ -0,0 +1,115 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train_256p'
+data_root_val = 'data/kinetics400/rawframes_val_256p'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes_256p.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes_256p.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_256p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=4, frame_interval=16, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.6, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/slowonly_r50_randomresizedcrop_256p_4x16x1'
+ '_256e_kinetics400_rgb')
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_320p_4x16x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_320p_4x16x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..24b12dc
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_320p_4x16x1_256e_kinetics400_rgb.py
@@ -0,0 +1,115 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train_320p'
+data_root_val = 'data/kinetics400/rawframes_val_320p'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes_320p.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=4, frame_interval=16, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.6, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/slowonly_r50_randomresizedcrop_320p_4x16x1'
+ '_256e_kinetics400_rgb')
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..97033fa
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/data_benchmark/slowonly_r50_randomresizedcrop_340x256_4x16x1_256e_kinetics400_rgb.py
@@ -0,0 +1,115 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=4, frame_interval=16, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.6, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('slowonly_r50_randomresizedcrop_320p_4x16x1'
+ '_256e_kinetics400_rgb')
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb.py
new file mode 100644
index 0000000..2cc9bf9
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb.py
@@ -0,0 +1,114 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/gym/subaction_frames'
+data_root_val = 'data/gym/subaction_frames'
+ann_file_train = 'data/gym/annotations/gym99_train_frame.txt'
+ann_file_val = 'data/gym/annotations/gym99_val_frame.txt'
+ann_file_test = 'data/gym/annotations/gym99_val_frame.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=4, frame_interval=16, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=24,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.03, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[90, 110])
+total_epochs = 120
+checkpoint_config = dict(interval=1)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowonly_imagenet_pretrained_r50_4x16x1_120e_gym99_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb.py
new file mode 100644
index 0000000..1409082
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_4x16x1_150e_kinetics400_rgb.py
@@ -0,0 +1,120 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=4, frame_interval=16, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='step',
+ step=[90, 130],
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=10)
+total_epochs = 150
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/slowonly_imagenet_pretrained_r50_4x16x1_150e'
+ '_kinetics400_rgb')
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb.py
new file mode 100644
index 0000000..1a2e208
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_imagenet_pretrained_r50_8x8x1_150e_kinetics400_rgb.py
@@ -0,0 +1,120 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='step',
+ step=[90, 130],
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=10)
+total_epochs = 150
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/slowonly_imagenet_pretrained_r50_8x8x1_150e'
+ '_kinetics400_rgb')
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow.py
new file mode 100644
index 0000000..61213d5
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow.py
@@ -0,0 +1,124 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ in_channels=2,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ with_pool2=False,
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/gym/subaction_frames'
+data_root_val = 'data/gym/subaction_frames'
+ann_file_train = 'data/gym/annotations/gym99_train_frame.txt'
+ann_file_val = 'data/gym/annotations/gym99_val_frame.txt'
+ann_file_test = 'data/gym/annotations/gym99_val_frame.txt'
+img_norm_cfg = dict(mean=[128, 128], std=[128, 128])
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=4, frame_interval=16, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=24,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ modality='Flow',
+ filename_tmpl='{}_{:05d}.jpg',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ modality='Flow',
+ filename_tmpl='{}_{:05d}.jpg',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ modality='Flow',
+ filename_tmpl='{}_{:05d}.jpg',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.03, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[90, 110])
+total_epochs = 120
+checkpoint_config = dict(interval=1)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/'
+ 'slowonly_kinetics_pretrained_r50_4x16x1_120e_gym99_flow')
+load_from = ('https://download.openmmlab.com/mmaction/recognition/slowonly/'
+ 'slowonly_r50_4x16x1_256e_kinetics400_flow/'
+ 'slowonly_r50_4x16x1_256e_kinetics400_flow_20200704-decb8568.pth')
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r101_8x8x1_196e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r101_8x8x1_196e_kinetics400_rgb.py
new file mode 100644
index 0000000..8b5ccbc
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r101_8x8x1_196e_kinetics400_rgb.py
@@ -0,0 +1,118 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=101,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.1, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr=0,
+ warmup='linear',
+ warmup_ratio=0.1,
+ warmup_by_epoch=True,
+ warmup_iters=34)
+total_epochs = 196
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowonly_r101_8x8x1_196e_kinetics400_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_flow.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_flow.py
new file mode 100644
index 0000000..78ae06d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_flow.py
@@ -0,0 +1,126 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ in_channels=2,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ with_pool2=False,
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics_flow_train_list.txt'
+ann_file_val = 'data/kinetics400/kinetics_flow_val_list.txt'
+ann_file_test = 'data/kinetics400/kinetics_flow_val_list.txt'
+img_norm_cfg = dict(mean=[128, 128], std=[128, 128])
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=4, frame_interval=16, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=24,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ modality='Flow',
+ filename_tmpl='{}_{:05d}.jpg',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ modality='Flow',
+ filename_tmpl='{}_{:05d}.jpg',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ modality='Flow',
+ filename_tmpl='{}_{:05d}.jpg',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.06, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr=0,
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=34)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowonly_r50_4x16x1_256e_kinetics400_flow'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..6125689
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_4x16x1_256e_kinetics400_rgb.py
@@ -0,0 +1,114 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=4, frame_interval=16, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.1, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowonly_r50_4x16x1_256e_kinetics400_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_flow.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_flow.py
new file mode 100644
index 0000000..d3f106e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_flow.py
@@ -0,0 +1,126 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ in_channels=2,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ with_pool2=False,
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics_flow_train_list.txt'
+ann_file_val = 'data/kinetics400/kinetics_flow_val_list.txt'
+ann_file_test = 'data/kinetics400/kinetics_flow_val_list.txt'
+img_norm_cfg = dict(mean=[128, 128], std=[128, 128])
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ modality='Flow',
+ filename_tmpl='{}_{:05d}.jpg',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ modality='Flow',
+ filename_tmpl='{}_{:05d}.jpg',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ modality='Flow',
+ filename_tmpl='{}_{:05d}.jpg',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.06, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr=0,
+ warmup='linear',
+ warmup_by_epoch=True,
+ warmup_iters=34)
+total_epochs = 196
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowonly_r50_8x8x1_256e_kinetics400_flow'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..65cd20d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_8x8x1_256e_kinetics400_rgb.py
@@ -0,0 +1,114 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.1, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowonly_r50_8x8x1_256e_kinetics400_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_4x16x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_4x16x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..97c15e0
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_4x16x1_256e_kinetics400_rgb.py
@@ -0,0 +1,117 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics400/videos_train'
+data_root_val = 'data/kinetics400/videos_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_videos.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_videos.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=4, frame_interval=16, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=24,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.3, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowonly_r50_video_4x16x1_256e_kinetics400_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics600_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics600_rgb.py
new file mode 100644
index 0000000..e91549b
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics600_rgb.py
@@ -0,0 +1,114 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=600,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics600/videos_train'
+data_root_val = 'data/kinetics600/videos_val'
+ann_file_train = 'data/kinetics600/kinetics600_train_list_videos.txt'
+ann_file_val = 'data/kinetics600/kinetics600_val_list_videos.txt'
+ann_file_test = 'data/kinetics600/kinetics600_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.15, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowonly_r50_video_8x8x1_256e_kinetics600_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics700_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics700_rgb.py
new file mode 100644
index 0000000..de7661b
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_8x8x1_256e_kinetics700_rgb.py
@@ -0,0 +1,114 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=700,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics700/videos_train'
+data_root_val = 'data/kinetics700/videos_val'
+ann_file_train = 'data/kinetics700/kinetics700_train_list_videos.txt'
+ann_file_val = 'data/kinetics700/kinetics700_val_list_videos.txt'
+ann_file_test = 'data/kinetics700/kinetics700_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='DecordDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.15, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 256
+checkpoint_config = dict(interval=4)
+workflow = [('train', 1)]
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/slowonly_r50_video_8x8x1_256e_kinetics700_rgb'
+load_from = None
+resume_from = None
+find_unused_parameters = False
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_inference_4x16x1_256e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_inference_4x16x1_256e_kinetics400_rgb.py
new file mode 100644
index 0000000..e2669a8
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/slowonly/slowonly_r50_video_inference_4x16x1_256e_kinetics400_rgb.py
@@ -0,0 +1,47 @@
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ cls_head=dict(
+ type='I3DHead',
+ in_channels=2048,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5))
+
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=4,
+ frame_interval=16,
+ num_clips=10,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/README.md
new file mode 100644
index 0000000..cca26b9
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/README.md
@@ -0,0 +1,75 @@
+# TIN
+
+## Introduction
+```
+@article{shao2020temporal,
+ title={Temporal Interlacing Network},
+ author={Hao Shao and Shengju Qian and Yu Liu},
+ year={2020},
+ journal={AAAI},
+}
+```
+
+## Model Zoo
+
+### Something-Something V1
+
+|config | resolution | gpus | backbone| pretrain | top1 acc| top5 acc | reference top1 acc | reference top5 acc | gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tin_r50_1x1x8_40e_sthv1_rgb](/configs/recognition/tin/tin_r50_1x1x8_40e_sthv1_rgb.py)|height 100|8x4| ResNet50 | ImageNet | 44.25 | 73.94 | 44.04 | 72.72 | 6181 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tin/tin_r50_1x1x8_40e_sthv1_rgb/tin_r50_1x1x8_40e_sthv1_rgb_20200729-4a33db86.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tin/tin_r50_1x1x8_40e_sthv1_rgb/20200729_034132.log) | [json](https://download.openmmlab.com/mmaction/recognition/tin/tin_r50_1x1x8_40e_sthv1_rgb/20200729_034132.log.json) |
+
+### Something-Something V2
+
+|config | resolution | gpus | backbone| pretrain | top1 acc| top5 acc | reference top1 acc | reference top5 acc | gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tin_r50_1x1x8_40e_sthv2_rgb](/configs/recognition/tin/tin_r50_1x1x8_40e_sthv2_rgb.py)|height 240|8x4| ResNet50 | ImageNet | 56.70 | 83.62 | 56.48 | 83.45 | 6185 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tin/tin_r50_1x1x8_40e_sthv2_rgb/tin_r50_1x1x8_40e_sthv2_rgb_20200912-b27a7337.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tin/tin_r50_1x1x8_40e_sthv2_rgb/20200912_225451.log) | [json](https://download.openmmlab.com/mmaction/recognition/tin/tin_r50_1x1x8_40e_sthv2_rgb/20200912_225451.log.json) |
+
+### Kinetics-400
+
+|config | resolution | gpus | backbone| pretrain | top1 acc| top5 acc | gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tin_tsm_finetune_r50_1x1x8_50e_kinetics400_rgb](/configs/recognition/tin/tin_tsm_finetune_r50_1x1x8_50e_kinetics400_rgb.py)|short-side 256|8x4| ResNet50 | TSM-Kinetics400 | 70.89 | 89.89 | 6187 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tin/tin_tsm_finetune_r50_1x1x8_50e_kinetics400_rgb/tin_tsm_finetune_r50_1x1x8_50e_kinetics400_rgb_20200810-4a146a70.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tin/tin_tsm_finetune_r50_1x1x8_50e_kinetics400_rgb/20200809_142447.log) | [json](https://download.openmmlab.com/mmaction/recognition/tin/tin_tsm_finetune_r50_1x1x8_50e_kinetics400_rgb/20200809_142447.log.json) |
+
+Here, we use `finetune` to indicate that we use [TSM model](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb/tsm_r50_1x1x8_50e_kinetics400_rgb_20200607-af7fb746.pth) trained on Kinetics-400 to finetune the TIN model on Kinetics-400.
+
+Notes:
+1. The **reference topk acc** are got by training the [original repo #1aacd0c](https://github.com/deepcs233/TIN/tree/1aacd0c4c30d5e1d334bf023e55b855b59f158db) with no [AverageMeter issue](https://github.com/deepcs233/TIN/issues/4).
+The [AverageMeter issue](https://github.com/deepcs233/TIN/issues/4) will lead to incorrect performance, so we fix it before running.
+2. The **gpus** indicates the number of gpu we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+3. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+4. The values in columns named after "reference" are the results got by training on the original repo, using the same model settings.
+
+For more details on data preparation, you can refer to Kinetics400, Something-Something V1 and Something-Something V2 in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train TIN model on Something-Something V1 dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/recognition/tin/tin_r50_1x1x8_40e_sthv1_rgb.py \
+ --work-dir work_dirs/tin_r50_1x1x8_40e_sthv1_rgb \
+ --validate --seed 0 --deterministic
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test TIN model on Something-Something V1 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/tin/tin_r50_1x1x8_40e_sthv1_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/tin_r50_1x1x8_40e_sthv1_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/tin_r50_1x1x8_40e_sthv1_rgb.py
new file mode 100644
index 0000000..2d40649
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/tin_r50_1x1x8_40e_sthv1_rgb.py
@@ -0,0 +1,131 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTIN',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=4),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.8,
+ init_std=0.001,
+ is_shift=False))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sth-v1/rawframes_train/'
+data_root_val = 'data/sth-v1/rawframes_val/'
+ann_file_train = 'data/sth-v1/sth-v1_train_list.txt'
+ann_file_val = 'data/sth-v1/sth-v1_val_list.txt'
+ann_file_test = 'data/sth-v1/sth-v1_val_list.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=6,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='{:05}.jpg',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.02, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0005)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ min_lr_ratio=0.5,
+ warmup='linear',
+ warmup_ratio=0.1,
+ warmup_by_epoch=True,
+ warmup_iters=1)
+total_epochs = 40
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tin_r50_1x1x8_40e_sthv1_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/tin_r50_1x1x8_40e_sthv2_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/tin_r50_1x1x8_40e_sthv2_rgb.py
new file mode 100644
index 0000000..1ace88a
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/tin_r50_1x1x8_40e_sthv2_rgb.py
@@ -0,0 +1,131 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTIN',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=4),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.8,
+ init_std=0.001,
+ is_shift=False))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sth-v2/rawframes_train/'
+data_root_val = 'data/sth-v2/rawframes_val/'
+ann_file_train = 'data/sth-v2/sth-v2_train_list.txt'
+ann_file_val = 'data/sth-v2/sth-v2_val_list.txt'
+ann_file_test = 'data/sth-v2/sth-v2_val_list.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=6,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='{:05}.jpg',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.02, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0005)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ by_epoch=False,
+ warmup='linear',
+ warmup_iters=1,
+ warmup_by_epoch=True,
+ min_lr=0)
+total_epochs = 40
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tin_r50_1x1x8_40e_sthv2_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/tin_tsm_finetune_r50_1x1x8_50e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/tin_tsm_finetune_r50_1x1x8_50e_kinetics400_rgb.py
new file mode 100644
index 0000000..0ad3c7b
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tin/tin_tsm_finetune_r50_1x1x8_50e_kinetics400_rgb.py
@@ -0,0 +1,124 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTIN',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=4),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=6,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.01, # this lr is for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tin_tsm_finetune_r50_1x1x8_50e_kinetics400_rgb/'
+# load_from = 'modelzoo/tsm_r50_1x1x8_50e_kinetics400_rgb_20200607-af7fb746.pth' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb/tsm_r50_1x1x8_50e_kinetics400_rgb_20200607-af7fb746.pth' # noqa: E501
+resume_from = None
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/README.md
new file mode 100644
index 0000000..b40ae97
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/README.md
@@ -0,0 +1,59 @@
+# TPN
+
+## Introduction
+```
+@inproceedings{yang2020tpn,
+ title={Temporal Pyramid Network for Action Recognition},
+ author={Yang, Ceyuan and Xu, Yinghao and Shi, Jianping and Dai, Bo and Zhou, Bolei},
+ booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
+ year={2020},
+}
+```
+
+## Model Zoo
+
+### Kinetics-400
+
+|config | resolution | gpus | backbone | pretrain | top1 acc| top5 acc | reference top1 acc | reference top5 acc | inference_time(video/s) | gpu_mem(M)| ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tpn_slowonly_r50_8x8x1_150e_kinetics_rgb](/configs/recognition/tpn/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb.py)|short-side 320|8x4| ResNet50 | ImageNet | 73.10 | 91.03 | x | x | x | 6916 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tpn/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb_20200910-b796d7a0.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tpn/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb/20200910_134330.log) | [json](https://download.openmmlab.com/mmaction/recognition/tpn/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb/20200910_134330.log.json) |
+|[tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb](/configs/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb.py)|short-side 320|8x4| ResNet50 | ImageNet | 76.20 | 92.44 | [75.49](https://github.com/decisionforce/TPN/blob/master/MODELZOO.md) | [92.05](https://github.com/decisionforce/TPN/blob/master/MODELZOO.md) | x | 6916 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/20200923_151919.log) | [json](https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/20200923_151919.log.json) |
+
+Notes:
+1. The **gpus** indicates the number of gpu we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+3. The values in columns named after "reference" are the results got by testing the checkpoint released on the original repo and codes, using the same dataset with ours.
+
+For more details on data preparation, you can refer to Kinetics400, Something-Something V1 and Something-Something V2 in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train TPN model on Kinetics-400 dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/recognition/tpn/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb.py \
+ --work-dir work_dirs/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb [--validate --seed 0 --deterministic]
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test TPN model on Kinetics-400 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/tpn/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json --average-clips prob
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/finetune_ucf101_tpn_slowonly_rpl.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/finetune_ucf101_tpn_slowonly_rpl.py
new file mode 100644
index 0000000..30499aa
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/finetune_ucf101_tpn_slowonly_rpl.py
@@ -0,0 +1,141 @@
+# model settings
+model = dict(
+ type='Recognizer3DRPL',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=101, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNRPLHead',
+ loss_cls=dict(type='RPLoss',
+ temperature=1,
+ weight_pl=0.1),
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.002, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly/finetune_ucf101_tpn_slowonly_rpl' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_bnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_bnn.py
new file mode 100644
index 0000000..b1fb8ac
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_bnn.py
@@ -0,0 +1,64 @@
+# model settings
+model = dict(
+ type='Recognizer3DBNN',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1)))),
+ cls_head=dict(
+ type='TPNBNNHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0, # for bnn, dropout is not necessary
+ init_std=0.01))
+test_cfg = dict(average_clips='prob', npass=10)
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=2,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_dnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_dnn.py
new file mode 100644
index 0000000..9ba1602
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_dnn.py
@@ -0,0 +1,65 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1)))),
+ cls_head=dict(
+ type='TPNHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+evidence='exp' # only used for EDL
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=2,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_enn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_enn.py
new file mode 100644
index 0000000..29e2c18
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_enn.py
@@ -0,0 +1,65 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1)))),
+ cls_head=dict(
+ type='TPNHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+evidence='exp' # only used for EDL
+test_cfg = dict(average_clips='score')
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=2,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_rpl.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_rpl.py
new file mode 100644
index 0000000..d488b45
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/inference_tpn_slowonly_rpl.py
@@ -0,0 +1,68 @@
+# model settings
+model = dict(
+ type='Recognizer3DRPL',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1)))),
+ cls_head=dict(
+ type='TPNRPLHead',
+ loss_cls=dict(type='RPLoss',
+ temperature=1,
+ weight_pl=0.1),
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+evidence='exp' # only used for EDL
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=2,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..6ba644e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,132 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=400, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='FrameSelector'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=8,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[75, 125])
+total_epochs = 150
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics400_rgb' # noqa: E501
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_bnn_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_bnn_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..75a9e55
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_bnn_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,139 @@
+# model settings
+model = dict(
+ type='Recognizer3DBNN',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=101, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNBNNHead',
+ loss_cls=dict(type='BayesianNNLoss'),
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0, # for bnn, dropout is not necessary
+ init_std=0.01))
+train_cfg = dict(loss_weight=1e-6, npass=2)
+test_cfg = dict(average_clips='prob', npass=10)
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.002, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly/finetune_ucf101_tpn_slowonly_bnn' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_celoss_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_celoss_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..f04ea0d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_celoss_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,138 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=101, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.002, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly/finetune_ucf101_tpn_slowonly_celoss' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_avuc_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_avuc_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..cdf439e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_avuc_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,146 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=101, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.002, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly/finetune_ucf101_tpn_slowonly_edlloss_avuc' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_avuc_debias_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_avuc_debias_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..71f2e01
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_avuc_debias_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,155 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=101, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01),
+ debias_head=dict(
+ type='DebiasHead',
+ loss_cls=evidence_loss, # actually not used!
+ loss_factor=0.1,
+ num_classes=101,
+ in_channels=1024,
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly/finetune_ucf101_tpn_slowonly_edlloss_avuc' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_avuc_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_avuc_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..dd53169
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_avuc_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,147 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=101, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly/finetune_ucf101_tpn_slowonly_edlloss_avuc' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_avuc_rebias_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_avuc_rebias_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..9ece651
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_avuc_rebias_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,149 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ rebias_head_cfg=dict(out_channels=101,
+ loss_weight=0.5,
+ loss_rebias=dict(type='RebiasLoss', lambda_g=1.0, criteria='hsic'))),
+ cls_head=dict(
+ type='TPNHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly/finetune_ucf101_tpn_slowonly_edlloss_nokl_avuc_rebias' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_davuc_debias_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_davuc_debias_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..48b0b4e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_davuc_debias_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,156 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ disentangle=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=101, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01),
+ debias_head=dict(
+ type='DebiasHead',
+ loss_cls=evidence_loss, # actually not used!
+ loss_factor=0.1,
+ num_classes=101,
+ in_channels=1024,
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly/finetune_ucf101_tpn_slowonly_edlloss_avuc' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..9e66ad1
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_nokl_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,147 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=False,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=101, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly/finetune_ucf101_tpn_slowonly_edlloss_avuc' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..810e4d4
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_edlloss_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,145 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ annealing_method='exp')
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained='torchvision://resnet50',
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=101, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='OpenCVDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.002, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly/finetune_ucf101_tpn_slowonly_edlloss' # noqa: E501
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tpn/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb/tpn_imagenet_pretrained_slowonly_r50_8x8x1_150e_kinetics_rgb_20200923-52629684.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb.py
new file mode 100644
index 0000000..6d66522
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_slowonly_r50_8x8x1_150e_kinetics_rgb.py
@@ -0,0 +1,132 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(
+ type='ResNet3dSlowOnly',
+ depth=50,
+ pretrained=None,
+ lateral=False,
+ out_indices=(2, 3),
+ conv1_kernel=(1, 7, 7),
+ conv1_stride_t=1,
+ pool1_stride_t=1,
+ inflate=(0, 0, 1, 1),
+ norm_eval=False),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=400, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='FrameSelector'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=1,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=8,
+ frame_interval=8,
+ num_clips=10,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=8,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[75, 125])
+total_epochs = 150
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_slowonly_r50_8x8x1_150e_kinetics400_rgb'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_tsm_r50_1x1x8_150e_sthv1_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_tsm_r50_1x1x8_150e_sthv1_rgb.py
new file mode 100644
index 0000000..3abf995
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tpn/tpn_tsm_r50_1x1x8_150e_sthv1_rgb.py
@@ -0,0 +1,128 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ out_indices=(2, 3),
+ norm_eval=False,
+ shift_div=8),
+ neck=dict(
+ type='TPN',
+ in_channels=(1024, 2048),
+ out_channels=1024,
+ spatial_modulation_cfg=dict(
+ in_channels=(1024, 2048), out_channels=2048),
+ temporal_modulation_cfg=dict(downsample_scales=(8, 8)),
+ upsample_cfg=dict(scale_factor=(1, 1, 1)),
+ downsample_cfg=dict(downsample_scale=(1, 1, 1)),
+ level_fusion_cfg=dict(
+ in_channels=(1024, 1024),
+ mid_channels=(1024, 1024),
+ out_channels=2048,
+ downsample_scales=((1, 1, 1), (1, 1, 1))),
+ aux_head_cfg=dict(out_channels=174, loss_weight=0.5)),
+ cls_head=dict(
+ type='TPNHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips=None)
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv1/rawframes'
+data_root_val = 'data/sthv1/rawframes'
+ann_file_train = 'data/sthv1/sthv1_train_list_rawframes.txt'
+ann_file_val = 'data/sthv1/sthv1_val_list_rawframes.txt'
+ann_file_test = 'data/sthv1/sthv1_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='FrameSelector'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='ColorJitter', color_space_aug=True),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=16,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=8,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0005,
+ nesterov=True) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[75, 125])
+total_epochs = 150
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tpn_tsm_r50_8x8x1_150e_kinetics400_rgb'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/README.md
new file mode 100644
index 0000000..4abd5b7
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/README.md
@@ -0,0 +1,115 @@
+# TSM
+
+## Introduction
+```
+@inproceedings{lin2019tsm,
+ title={TSM: Temporal Shift Module for Efficient Video Understanding},
+ author={Lin, Ji and Gan, Chuang and Han, Song},
+ booktitle={Proceedings of the IEEE International Conference on Computer Vision},
+ year={2019}
+}
+
+@article{NonLocal2018,
+ author = {Xiaolong Wang and Ross Girshick and Abhinav Gupta and Kaiming He},
+ title = {Non-local Neural Networks},
+ journal = {CVPR},
+ year = {2018}
+}
+```
+
+## Model Zoo
+
+### Kinetics-400
+
+|config | resolution | gpus | backbone | pretrain | top1 acc| top5 acc | reference top1 acc | reference top5 acc | inference_time(video/s) | gpu_mem(M)| ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsm_r50_1x1x8_50e_kinetics400_rgb](/configs/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb.py) |340x256|8| ResNet50| ImageNet |70.24|89.56|[70.36](https://github.com/mit-han-lab/temporal-shift-module/blob/8d53d6fda40bea2f1b37a6095279c4b454d672bd/scripts/train_tsm_kinetics_rgb_8f.sh)|[89.49](https://github.com/mit-han-lab/temporal-shift-module/blob/8d53d6fda40bea2f1b37a6095279c4b454d672bd/scripts/train_tsm_kinetics_rgb_8f.sh)|74.0 (8x1 frames)| 7079 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb/tsm_r50_1x1x8_50e_kinetics400_rgb_20200607-af7fb746.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb/20200607_211800.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb/20200607_211800.log.json)|
+|[tsm_r50_1x1x8_50e_kinetics400_rgb](/configs/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb.py) |short-side 256|8| ResNet50| ImageNet |70.59|89.52|x|x|x|7079|[ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_256p_1x1x8_50e_kinetics400_rgb/tsm_r50_256p_1x1x8_50e_kinetics400_rgb_20200726-020785e2.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_256p_1x1x8_50e_kinetics400_rgb/20200725_031623.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_256p_1x1x8_50e_kinetics400_rgb/20200725_031623.log.json)|
+|[tsm_r50_video_1x1x8_50e_kinetics400_rgb](/configs/recognition/tsm/tsm_r50_video_1x1x8_50e_kinetics400_rgb.py) |short-side 256|8| ResNet50| ImageNet |70.25|89.66|[70.36](https://github.com/mit-han-lab/temporal-shift-module/blob/8d53d6fda40bea2f1b37a6095279c4b454d672bd/scripts/train_tsm_kinetics_rgb_8f.sh)|[89.49](https://github.com/mit-han-lab/temporal-shift-module/blob/8d53d6fda40bea2f1b37a6095279c4b454d672bd/scripts/train_tsm_kinetics_rgb_8f.sh)|74.0 (8x1 frames)| 7077 | [ckpt]( https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_video_1x1x8_100e_kinetics400_rgb/tsm_r50_video_1x1x8_100e_kinetics400_rgb_20200702-a77f4328.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_video_1x1x8_100e_kinetics400_rgb/tsm_r50_video_2d_1x1x8_50e_kinetics400_rgb.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_video_1x1x8_100e_kinetics400_rgb/tsm_r50_video_2d_1x1x8_50e_kinetics400_rgb.log.json)|
+|[tsm_r50_dense_1x1x8_100e_kinetics400_rgb](/configs/recognition/tsm/tsm_r50_dense_1x1x8_100e_kinetics400_rgb.py) |340x256|8x4| ResNet50 | ImageNet|72.9|90.44|[72.22](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#dense-sample)|[90.37](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#dense-sample)|11.5 (8x10 frames)| 7079 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_dense_1x1x8_100e_kinetics400_rgb/tsm_r50_dense_1x1x8_100e_kinetics400_rgb_20200626-91a54551.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_dense_1x1x8_100e_kinetics400_rgb/20200626_213415.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_dense_1x1x8_100e_kinetics400_rgb/20200626_213415.log.json)|
+|[tsm_r50_dense_1x1x8_100e_kinetics400_rgb](/configs/recognition/tsm/tsm_r50_dense_1x1x8_100e_kinetics400_rgb.py) |short-side 256|8| ResNet50 | ImageNet|73.38|91.02|x|x|x|7079|[ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb_20200727-e1e0c785.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb/20200725_032043.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb/20200725_032043.log.json)|
+|[tsm_r50_1x1x16_50e_kinetics400_rgb](/configs/recognition/tsm/tsm_r50_1x1x16_50e_kinetics400_rgb.py) |340x256|8| ResNet50| ImageNet |72.09|90.37|[70.67](https://github.com/mit-han-lab/temporal-shift-module/blob/8d53d6fda40bea2f1b37a6095279c4b454d672bd/scripts/train_tsm_kinetics_rgb_16f.sh)|[89.98](https://github.com/mit-han-lab/temporal-shift-module/blob/8d53d6fda40bea2f1b37a6095279c4b454d672bd/scripts/train_tsm_kinetics_rgb_16f.sh)|47.0 (16x1 frames)| 10404 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x16_50e_kinetics400_rgb/tsm_r50_340x256_1x1x16_50e_kinetics400_rgb_20201011-2f27f229.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x16_50e_kinetics400_rgb/20201011_205356.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x16_50e_kinetics400_rgb/20201011_205356.log.json)|
+|[tsm_r50_1x1x16_50e_kinetics400_rgb](/configs/recognition/tsm/tsm_r50_1x1x16_50e_kinetics400_rgb.py) |short-side 256|8x4| ResNet50| ImageNet |71.89|90.73|x|x|x|10398|[ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_256p_1x1x16_50e_kinetics400_rgb/tsm_r50_256p_1x1x16_50e_kinetics400_rgb_20201010-85645c2a.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_256p_1x1x16_50e_kinetics400_rgb/20201010_224825.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_256p_1x1x16_50e_kinetics400_rgb/20201010_224825.log.json)|
+|[tsm_nl_embedded_gaussian_r50_1x1x8_50e_kinetics400_rgb](/configs/recognition/tsm/tsm_nl_embedded_gaussian_r50_1x1x8_50e_kinetics400_rgb.py)|short-side 320|8x4| ResNet50| ImageNet |72.03|90.25|71.81|90.36|x|8931|[ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_nl_embedded_gaussian_r50_1x1x8_50e_kinetics400_rgb/tsm_nl_embedded_gaussian_r50_1x1x8_50e_kinetics400_rgb_20200724-f00f1336.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_nl_embedded_gaussian_r50_1x1x8_50e_kinetics400_rgb/20200724_120023.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_nl_embedded_gaussian_r50_1x1x8_50e_kinetics400_rgb/20200724_120023.log.json)|
+|[tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb](/configs/recognition/tsm/tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb.py)|short-side 320|8x4| ResNet50| ImageNet |70.70|89.90|x|x|x|10125|[ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb/tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb_20200816-b93fd297.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb/20200815_210253.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb/20200815_210253.log.json)|
+|[tsm_nl_dot_product_r50_1x1x8_50e_kinetics400_rgb](/configs/recognition/tsm/tsm_nl_dot_product_r50_1x1x8_50e_kinetics400_rgb.py)|short-side 320|8x4|ResNet50| ImageNet |71.60|90.34|x|x|x|8358|[ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_nl_dot_product_r50_1x1x8_50e_kinetics400_rgb/tsm_nl_dot_product_r50_1x1x8_50e_kinetics400_rgb_20200724-d8ad84d2.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_nl_dot_product_r50_1x1x8_50e_kinetics400_rgb/20200723_220442.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_nl_dot_product_r50_1x1x8_50e_kinetics400_rgb/20200723_220442.log.json)|
+
+### Something-Something V1
+
+|config | resolution | gpus | backbone| pretrain | top1 acc (efficient/accurate)| top5 acc (efficient/accurate)| reference top1 acc (efficient/accurate)| reference top5 acc (efficient/accurate)| gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsm_r50_1x1x8_50e_sthv1_rgb](/configs/recognition/tsm/tsm_r50_1x1x8_50e_sthv1_rgb.py) |height 100|8| ResNet50 | ImageNet|45.46 / 47.21|74.71 / 76.09|[45.50 / 47.33](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[74.34 / 76.60](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)| 7077| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_sthv1_rgb/tsm_r50_1x1x8_50e_sthv1_rgb_20200616-3417f361.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_sthv1_rgb/20200616_022852.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_sthv1_rgb/20200616_022852.log.json)|
+|[tsm_r50_1x1x16_50e_sthv1_rgb](/configs/recognition/tsm/tsm_r50_1x1x16_50e_sthv1_rgb.py)|height 100|8| ResNet50 | ImageNet|47.62 / 49.28|76.63 / 77.82|[47.05 / 48.61](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[76.40 / 77.96](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|10390|[ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x16_50e_sthv1_rgb/tsm_r50_1x1x16_50e_sthv1_rgb_20201010-17fa49f6.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x16_50e_sthv1_rgb/20201010_221240.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x16_50e_sthv1_rgb/20201010_221240.log.json)|
+|[tsm_r101_1x1x8_50e_sthv1_rgb](/configs/recognition/tsm/tsm_r101_1x1x8_50e_sthv1_rgb.py)|height 100|8| ResNet50 | ImageNet|45.72 / 48.43|74.67 / 76.72|[46.64 / 48.13](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[75.40 / 77.31](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|9800|[ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r101_1x1x8_50e_sthv1_rgb/tsm_r101_1x1x8_50e_sthv1_rgb_20201010-43fedf2e.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r101_1x1x8_50e_sthv1_rgb/20201010_224055.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r101_1x1x8_50e_sthv1_rgb/20201010_224055.log.json)|
+
+### Something-Something V2
+
+|config | resolution | gpus | backbone | pretrain| top1 acc (efficient/accurate)| top5 acc (efficient/accurate)| reference top1 acc (efficient/accurate)| reference top5 acc (efficient/accurate)| gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsm_r50_1x1x8_50e_sthv2_rgb](/configs/recognition/tsm/tsm_r50_1x1x8_50e_sthv2_rgb.py) |height 240|8| ResNet50| ImageNet |57.86 / 61.12|84.67 / 86.26|[57.98 / 60.69](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[84.57 / 86.28](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)| 7069 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_sthv2_rgb/tsm_r50_1x1x8_50e_sthv2_rgb_20200912-033c4ac6.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_sthv2_rgb/20200912_140737.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x8_50e_sthv2_rgb/20200912_140737.log.json)|
+|[tsm_r50_1x1x16_50e_sthv2_rgb](/configs/recognition/tsm/tsm_r50_1x1x16_50e_sthv2_rgb.py) |height 240|8| ResNet50| ImageNet |59.93 / 62.04|86.10 / 87.35|[58.90 / 60.98](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[85.29 / 86.60](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)| 10400| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x16_50e_sthv2_rgb/tsm_r50_1x1x16_50e_sthv2_rgb_20201010-16469c6f.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x16_50e_sthv2_rgb/20201010_224215.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_1x1x16_50e_sthv2_rgb/20201010_224215.log.json)|
+|[tsm_r101_1x1x8_50e_sthv2_rgb](/configs/recognition/tsm/tsm_r101_1x1x8_50e_sthv2_rgb.py) |height 240|8| ResNet101 | ImageNet|58.59 / 61.51|85.07 / 86.90|[58.89 / 61.36](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[85.14 / 87.00](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)| 9784 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r101_1x1x8_50e_sthv2_rgb/tsm_r101_1x1x8_50e_sthv2_rgb_20201010-98cdedb8.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r101_1x1x8_50e_sthv2_rgb/20201010_224100.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r101_1x1x8_50e_sthv2_rgb/20201010_224100.log.json)|
+
+Notes:
+1. The **gpus** indicates the number of gpu we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+3. The values in columns named after "reference" are the results got by training on the original repo, using the same model settings. The checkpoints for reference repo can be downloaded [here](https://download.openmmlab.com/mmaction/recognition/tsm/tsm_reference_ckpt.rar).
+4. There are two kinds of test settings for Something-Something dataset, efficient setting (center crop * 1 clip) and accurate setting (Three crop * 2 clip), which is referred from the [original repo](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd).
+We use efficient setting as default provided in config files, and it can be changed to accurate setting by
+```python
+...
+test_cfg = dict(average_clips='prob')
+...
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=16, # `num_clips = 8` when using 8 segments
+ twice_sample=True, # set `twice_sample=True` for twice sample in accurate setting
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ # dict(type='CenterCrop', crop_size=224), it is used for efficient setting
+ dict(type='ThreeCrop', crop_size=256), # it is used for accurate setting
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+```
+
+For more details on data preparation, you can refer to Kinetics400, Something-Something V1 and Something-Something V2 in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train TSM model on Kinetics-400 dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb.py \
+ --work-dir work_dirs/tsm_r50_1x1x8_100e_kinetics400_rgb \
+ --validate --seed 0 --deterministic
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test TSM model on Kinetics-400 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_bnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_bnn.py
new file mode 100644
index 0000000..2589b50
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_bnn.py
@@ -0,0 +1,131 @@
+# model settings
+model = dict(
+ type='Recognizer2DBNN',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMBNNHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = dict(loss_weight=1e-6, npass=2)
+test_cfg = dict(average_clips='prob', npass=10)
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ val_dataloader=dict(videos_per_gpu=4),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ lr=0.001, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm/finetune_ucf101_tsm_bnn'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb_20200727-e1e0c785.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_dnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_dnn.py
new file mode 100644
index 0000000..498038e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_dnn.py
@@ -0,0 +1,133 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ val_dataloader=dict(videos_per_gpu=4),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.001, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm/finetune_ucf101_tsm_dnn'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb_20200727-e1e0c785.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_edlnokl_avuc_debias.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_edlnokl_avuc_debias.py
new file mode 100644
index 0000000..80d368d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_edlnokl_avuc_debias.py
@@ -0,0 +1,150 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True),
+ debias_head=dict(
+ type='DebiasHead',
+ loss_cls=evidence_loss, # actually not used!
+ loss_factor=0.1,
+ num_classes=101,
+ in_channels=2048,
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ val_dataloader=dict(videos_per_gpu=4),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.001, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm/finetune_ucf101_tsm_edlnokl_avuc_debias'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb_20200727-e1e0c785.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_rpl.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_rpl.py
new file mode 100644
index 0000000..c748e73
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/finetune_ucf101_tsm_rpl.py
@@ -0,0 +1,134 @@
+# model settings
+model = dict(
+ type='Recognizer2DRPL',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMRPLHead',
+ loss_cls=dict(type='RPLoss',
+ temperature=1,
+ weight_pl=0.1),
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/ucf101/videos'
+data_root_val = 'data/ucf101/videos'
+ann_file_train = 'data/ucf101/ucf101_train_split_1_videos.txt'
+ann_file_val = 'data/ucf101/ucf101_val_split_1_videos.txt'
+ann_file_test = 'data/ucf101/ucf101_val_split_1_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ val_dataloader=dict(videos_per_gpu=4),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ lr=0.001, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm/finetune_ucf101_tsm_rpl'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tsm/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb/tsm_r50_dense_256p_1x1x8_100e_kinetics400_rgb_20200727-e1e0c785.pth'
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_bnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_bnn.py
new file mode 100644
index 0000000..940b7e5
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_bnn.py
@@ -0,0 +1,50 @@
+# model settings
+model = dict(
+ type='Recognizer2DBNN',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMBNNHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+test_cfg = dict(average_clips='prob', npass=10)
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ start_index=0,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_dnn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_dnn.py
new file mode 100644
index 0000000..a392cd4
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_dnn.py
@@ -0,0 +1,50 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ start_index=0,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_enn.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_enn.py
new file mode 100644
index 0000000..7880afb
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_enn.py
@@ -0,0 +1,56 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ loss_cls=dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ annealing_method='exp'),
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+evidence='exp' # only used for EDL
+test_cfg = dict(average_clips='score')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ start_index=0,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_rpl.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_rpl.py
new file mode 100644
index 0000000..bdeb701
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/inference_tsm_rpl.py
@@ -0,0 +1,53 @@
+# model settings
+model = dict(
+ type='Recognizer2DRPL',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMRPLHead',
+ loss_cls=dict(type='RPLoss',
+ temperature=1,
+ weight_pl=0.1),
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ start_index=0,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/train_kinetics10_tsm_DEAR.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/train_kinetics10_tsm_DEAR.py
new file mode 100644
index 0000000..f73bb21
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/train_kinetics10_tsm_DEAR.py
@@ -0,0 +1,150 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True),
+ debias_head=dict(
+ type='DebiasHead',
+ loss_cls=evidence_loss, # actually not used!
+ loss_factor=0.1,
+ num_classes=101,
+ in_channels=2048,
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics10/videos_train'
+data_root_val = 'data/kinetics10/videos_val'
+ann_file_train = 'data/kinetics10/kinetics10_train_list_videos.txt'
+ann_file_val = 'data/kinetics10/kinetics10_val_list_videos.txt'
+ann_file_test = 'data/kinetics10/kinetics10_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=4,
+ workers_per_gpu=4,
+ val_dataloader=dict(videos_per_gpu=4),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.001, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm/train_kinetics10_tsm_DEAR'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/train_kinetics10_tsm_DEAR_noDebias.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/train_kinetics10_tsm_DEAR_noDebias.py
new file mode 100644
index 0000000..e51ffef
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/train_kinetics10_tsm_DEAR_noDebias.py
@@ -0,0 +1,142 @@
+# model settings
+evidence_loss = dict(type='EvidenceLoss',
+ num_classes=101,
+ evidence='exp',
+ loss_type='log',
+ with_kldiv=False,
+ with_avuloss=True,
+ annealing_method='exp')
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ loss_cls=evidence_loss,
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='evidence', evidence_type='exp')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics10/videos_train'
+data_root_val = 'data/kinetics10/videos_val'
+ann_file_train = 'data/kinetics10/kinetics10_train_list_videos.txt'
+ann_file_val = 'data/kinetics10/kinetics10_val_list_videos.txt'
+ann_file_test = 'data/kinetics10/kinetics10_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(type='DenseSampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=4,
+ workers_per_gpu=4,
+ val_dataloader=dict(videos_per_gpu=4),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.001, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=10)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook'),
+ ])
+annealing_runner = True
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm/train_kinetics10_tsm_DEAR_noDebias'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_nl_dot_product_r50_1x1x8_50e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_nl_dot_product_r50_1x1x8_50e_kinetics400_rgb.py
new file mode 100644
index 0000000..e59c8ec
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_nl_dot_product_r50_1x1x8_50e_kinetics400_rgb.py
@@ -0,0 +1,132 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ non_local=((0, 0, 0), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 0, 0)),
+ non_local_cfg=dict(
+ sub_sample=True,
+ use_scale=False,
+ norm_cfg=dict(type='BN3d', requires_grad=True),
+ mode='dot_product'),
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.01, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_nl_embedded_gaussian_r50_1x1x8_50e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_nl_embedded_gaussian_r50_1x1x8_50e_kinetics400_rgb.py
new file mode 100644
index 0000000..4253371
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_nl_embedded_gaussian_r50_1x1x8_50e_kinetics400_rgb.py
@@ -0,0 +1,132 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ non_local=((0, 0, 0), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 0, 0)),
+ non_local_cfg=dict(
+ sub_sample=True,
+ use_scale=False,
+ norm_cfg=dict(type='BN3d', requires_grad=True),
+ mode='embedded_gaussian'),
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.01, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_nl_embedded_gaussian_r50_1x1x8_50e_kinetics400_rgb/' # noqa: E501
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb.py
new file mode 100644
index 0000000..9a4d25b
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb.py
@@ -0,0 +1,132 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ non_local=((0, 0, 0), (1, 0, 1, 0), (1, 0, 1, 0, 1, 0), (0, 0, 0)),
+ non_local_cfg=dict(
+ sub_sample=True,
+ use_scale=False,
+ norm_cfg=dict(type='BN3d', requires_grad=True),
+ mode='gaussian'),
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.01, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_nl_gaussian_r50_1x1x8_50e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r101_1x1x8_50e_sthv1_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r101_1x1x8_50e_sthv1_rgb.py
new file mode 100644
index 0000000..fadb840
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r101_1x1x8_50e_sthv1_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet101',
+ depth=101,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv1/rawframes'
+data_root_val = 'data/sthv1/rawframes'
+ann_file_train = 'data/sthv1/sthv1_train_list_rawframes.txt'
+ann_file_val = 'data/sthv1/sthv1_val_list_rawframes.txt'
+ann_file_test = 'data/sthv1/sthv1_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='{:05}.jpg',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.01, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0005)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_r101_1x1x8_50e_sthv1_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r101_1x1x8_50e_sthv2_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r101_1x1x8_50e_sthv2_rgb.py
new file mode 100644
index 0000000..847f290
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r101_1x1x8_50e_sthv2_rgb.py
@@ -0,0 +1,123 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet101',
+ depth=101,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv2/rawframes'
+data_root_val = 'data/sthv2/rawframes'
+ann_file_train = 'data/sthv2/sthv2_train_list_rawframes.txt'
+ann_file_val = 'data/sthv2/sthv2_val_list_rawframes.txt'
+ann_file_test = 'data/sthv2/sthv2_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.01, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0005)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_r101_1x1x8_50e_sthv2_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x16_50e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x16_50e_kinetics400_rgb.py
new file mode 100644
index 0000000..be61619
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x16_50e_kinetics400_rgb.py
@@ -0,0 +1,128 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ num_segments=16,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=400,
+ num_segments=16,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=16),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=16,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=16,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=6,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.0075, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_r50_1x1x16_50e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x16_50e_sthv1_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x16_50e_sthv1_rgb.py
new file mode 100644
index 0000000..a571b8a
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x16_50e_sthv1_rgb.py
@@ -0,0 +1,128 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ num_segments=16,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=174,
+ num_segments=16,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv1/rawframes'
+data_root_val = 'data/sthv1/rawframes'
+ann_file_train = 'data/sthv1/sthv1_train_list_rawframes.txt'
+ann_file_val = 'data/sthv1/sthv1_val_list_rawframes.txt'
+ann_file_test = 'data/sthv1/sthv1_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=16),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=16,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=16,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=6,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='{:05}.jpg',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.0075, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0005)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_r50_1x1x16_50e_sthv1_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x16_50e_sthv2_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x16_50e_sthv2_rgb.py
new file mode 100644
index 0000000..e9bae6b
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x16_50e_sthv2_rgb.py
@@ -0,0 +1,125 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ num_segments=16,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=174,
+ num_segments=16,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv2/rawframes'
+data_root_val = 'data/sthv2/rawframes'
+ann_file_train = 'data/sthv2/sthv2_train_list_rawframes.txt'
+ann_file_val = 'data/sthv2/sthv2_val_list_rawframes.txt'
+ann_file_test = 'data/sthv2/sthv2_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=16),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=16,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=16,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=6,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.0075, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0005)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_r50_1x1x16_50e_sthv2_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb.py
new file mode 100644
index 0000000..d737207
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.01, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_r50_1x1x8_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x8_50e_sthv1_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x8_50e_sthv1_rgb.py
new file mode 100644
index 0000000..5f6f895
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x8_50e_sthv1_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv1/rawframes'
+data_root_val = 'data/sthv1/rawframes'
+ann_file_train = 'data/sthv1/sthv1_train_list_rawframes.txt'
+ann_file_val = 'data/sthv1/sthv1_val_list_rawframes.txt'
+ann_file_test = 'data/sthv1/sthv1_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='{:05}.jpg',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.01, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0005)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_r50_1x1x8_50e_sthv1_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x8_50e_sthv2_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x8_50e_sthv2_rgb.py
new file mode 100644
index 0000000..1738648
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_1x1x8_50e_sthv2_rgb.py
@@ -0,0 +1,123 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv2/rawframes'
+data_root_val = 'data/sthv2/rawframes'
+ann_file_train = 'data/sthv2/sthv2_train_list_rawframes.txt'
+ann_file_val = 'data/sthv2/sthv2_val_list_rawframes.txt'
+ann_file_test = 'data/sthv2/sthv2_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=16,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=16,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=6,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.0075, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0005)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_r50_1x1x8_50e_sthv2_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_dense_1x1x8_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_dense_1x1x8_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..f1470ba
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_dense_1x1x8_100e_kinetics400_rgb.py
@@ -0,0 +1,127 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DenseSampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ val_dataloader=dict(videos_per_gpu=4),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.02, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_r50_dense_1x1x8_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_video_1x1x8_50e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_video_1x1x8_50e_kinetics400_rgb.py
new file mode 100644
index 0000000..a936169
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_video_1x1x8_50e_kinetics400_rgb.py
@@ -0,0 +1,129 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics400/videos_train'
+data_root_val = 'data/kinetics400/videos_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_videos.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_videos.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.02, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_r50_video_2d_1x1x8_50e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_video_inference_1x1x8_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_video_inference_1x1x8_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..e09f5f1
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_r50_video_inference_1x1x8_100e_kinetics400_rgb.py
@@ -0,0 +1,49 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ is_shift=True))
+# model training and testing settings
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='DecordInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_temporal_pool_r50_1x1x8_50e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_temporal_pool_r50_1x1x8_50e_kinetics400_rgb.py
new file mode 100644
index 0000000..5d79eaf
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsm/tsm_temporal_pool_r50_1x1x8_50e_kinetics400_rgb.py
@@ -0,0 +1,128 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ temporal_pool=True,
+ shift_div=8),
+ cls_head=dict(
+ type='TSMHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001,
+ temporal_pool=True,
+ is_shift=True))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.01, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsm_temporal_pool_r50_1x1x8_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/README.md
new file mode 100644
index 0000000..d1749f4
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/README.md
@@ -0,0 +1,200 @@
+# TSN
+
+## Introduction
+```
+@inproceedings{wang2016temporal,
+ title={Temporal segment networks: Towards good practices for deep action recognition},
+ author={Wang, Limin and Xiong, Yuanjun and Wang, Zhe and Qiao, Yu and Lin, Dahua and Tang, Xiaoou and Van Gool, Luc},
+ booktitle={European conference on computer vision},
+ pages={20--36},
+ year={2016},
+ organization={Springer}
+}
+```
+
+## Model Zoo
+
+### UCF-101
+
+|config | gpus | backbone | pretrain | top1 acc| top5 acc | gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsn_r50_1x1x3_75e_ucf101_rgb](/configs/recognition/tsn/tsn_r50_1x1x3_75e_ucf101_rgb.py) [1] |8| ResNet50 | ImageNet |83.03|96.78|8332| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_75e_ucf101_rgb/tsn_r50_1x1x3_75e_ucf101_rgb_20201023-d85ab600.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_75e_ucf101_rgb/tsn_r50_1x1x3_75e_ucf101_rgb_20201023.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_75e_ucf101_rgb/tsn_r50_1x1x3_75e_ucf101_rgb_20201023.json) |
+
+[1] We report the performance on UCF-101 split1.
+### HMDB51
+
+|config | gpus | backbone | pretrain | top1 acc| top5 acc | gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsn_r50_1x1x8_50e_hmdb51_imagenet_rgb](/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_imagenet_rgb.py)|8| ResNet50 | ImageNet | 48.95| 80.19| 21535| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_imagenet_rgb/tsn_r50_1x1x8_50e_hmdb51_imagenet_rgb_20201123-ce6c27ed.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_imagenet_rgb/20201025_231108.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_imagenet_rgb/20201025_231108.log.json) |
+|[tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb.py) |8| ResNet50 | Kinetics400 | 56.08 | 84.31 | 21535| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb_20201123-7f84701b.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb/20201108_190805.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb/20201108_190805.log.json) |
+|[tsn_r50_1x1x8_50e_hmdb51_mit_rgb](/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_mit_rgb.py) |8| ResNet50 | Moments | 54.25 | 83.86| 21535| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_mit_rgb/tsn_r50_1x1x8_50e_hmdb51_mit_rgb_20201123-01526d41.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_mit_rgb/20201112_170135.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_mit_rgb/20201112_170135.log.json) |
+
+
+### Kinetics-400
+
+|config | resolution | gpus | backbone|pretrain | top1 acc| top5 acc | reference top1 acc | reference top5 acc | inference_time(video/s) | gpu_mem(M)| ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsn_r50_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py) |340x256|8| ResNet50 | ImageNet|70.60|89.26|x|x|4.3 (25x10 frames)|8344| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/20200614_063526.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/20200614_063526.log.json)|
+|[tsn_r50_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py) |short-side 256|8| ResNet50 | ImageNet|70.42|89.03|x|x|x|8343|[ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_1x1x3_100e_kinetics400_rgb/tsn_r50_256p_1x1x3_100e_kinetics400_rgb_20200725-22592236.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_1x1x3_100e_kinetics400_rgb/20200725_031325.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_1x1x3_100e_kinetics400_rgb/20200725_031325.log.json)|
+|[tsn_r50_dense_1x1x5_50e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_dense_1x1x5_100e_kinetics400_rgb.py) |340x256|8x3| ResNet50| ImageNet |70.18|89.10|[69.15](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[88.56](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|12.7 (8x10 frames)|7028| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_dense_1x1x5_100e_kinetics400_rgb/tsn_r50_dense_1x1x5_100e_kinetics400_rgb_20200627-a063165f.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_dense_1x1x5_100e_kinetics400_rgb/20200627_105310.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_dense_1x1x5_100e_kinetics400_rgb/20200627_105310.log.json)|
+|[tsn_r50_320p_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_320p_1x1x3_100e_kinetics400_rgb.py) |short-side 320|8x2| ResNet50| ImageNet |70.91|89.51|x|x|10.7 (25x3 frames)| 8344 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_320p_1x1x3_100e_kinetics400_rgb_20200702-cc665e2a.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_f3_kinetics400_shortedge_70.9_89.5.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_f3_kinetics400_shortedge_70.9_89.5.log.json)|
+|[tsn_r50_320p_1x1x3_110e_kinetics400_flow](/configs/recognition/tsn/tsn_r50_320p_1x1x3_110e_kinetics400_flow.py) |short-side 320|8x2| ResNet50 | ImageNet|55.70|79.85|x|x|x| 8471 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x3_110e_kinetics400_flow/tsn_r50_320p_1x1x3_110e_kinetics400_flow_20200705-3036bab6.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x3_110e_kinetics400_flow/tsn_r50_f3_kinetics400_flow_shortedge_55.7_79.9.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x3_110e_kinetics400_flow/tsn_r50_f3_kinetics400_flow_shortedge_55.7_79.9.log.json)|
+|tsn_r50_320p_1x1x3_kinetics400_twostream [1: 1]* |x|x| ResNet50 | ImageNet|72.76|90.52| x | x | x | x | x|x|x|
+|[tsn_r50_1x1x8_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_320p_1x1x8_100e_kinetics400_rgb.py)|short-side 256|8| ResNet50| ImageNet |71.80|90.17|x|x|x|8343|[ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_1x1x8_100e_kinetics400_rgb/tsn_r50_256p_1x1x8_100e_kinetics400_rgb_20200817-883baf16.pth)|[log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_1x1x8_100e_kinetics400_rgb/20200815_173413.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_1x1x8_100e_kinetics400_rgb/20200815_173413.log.json)|
+|[tsn_r50_320p_1x1x8_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_320p_1x1x8_100e_kinetics400_rgb.py) |short-side 320|8x3| ResNet50| ImageNet |72.41|90.55|x|x|11.1 (25x3 frames)| 8344 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_100e_kinetics400_rgb/tsn_r50_320p_1x1x8_100e_kinetics400_rgb_20200702-ef80e3d7.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_100e_kinetics400_rgb/tsn_r50_f8_kinetics400_shortedge_72.4_90.6.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_100e_kinetics400_rgb/tsn_r50_f8_kinetics400_shortedge_72.4_90.6.log.json)|
+|[tsn_r50_320p_1x1x8_110e_kinetics400_flow](/configs/recognition/tsn/tsn_r50_320p_1x1x8_110e_kinetics400_flow.py) |short-side 320|8x4| ResNet50 | ImageNet|57.76|80.99|x|x|x| 8473 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_110e_kinetics400_flow/tsn_r50_320p_1x1x8_110e_kinetics400_flow_20200705-1f39486b.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_110e_kinetics400_flow/tsn_r50_f8_kinetics400_flow_shortedge_57.8_81.0.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_110e_kinetics400_flow/tsn_r50_f8_kinetics400_flow_shortedge_57.8_81.0.log.json)|
+|tsn_r50_320p_1x1x8_kinetics400_twostream [1: 1]* |x|x| ResNet50| ImageNet |74.64|91.77| x | x | x | x | x|x|x|
+|[tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb.py) |short-side 320|8| ResNet50 | ImageNet |71.11|90.04| x | x | x | 8343 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb_20201014-5ae1ee79.pth) |[log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb_20201014.log)|[json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb_20201014.json)|
+|[tsn_r50_dense_1x1x8_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_dense_1x1x8_100e_kinetics400_rgb.py) |340x256|8| ResNet50 | ImageNet|70.77|89.3|[68.75](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[88.42](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|12.2 (8x10 frames)|8344| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_dense_1x1x8_100e_kinetics400_rgb/tsn_r50_dense_1x1x8_100e_kinetics400_rgb_20200606-e925e6e3.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_dense_1x1x8_100e_kinetics400_rgb/20200606_003901.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_dense_1x1x8_100e_kinetics400_rgb/20200606_003901.log.json)|
+|[tsn_r50_video_1x1x8_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics400_rgb.py) |short-side 256|8| ResNet50| ImageNet | 71.79 | 90.25 |x|x|x|21558| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics400_rgb/tsn_r50_video_1x1x8_100e_kinetics400_rgb_20200702-568cde33.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics400_rgb/tsn_r50_video_2d_1x1x8_100e_kinetics400_rgb.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics400_rgb/tsn_r50_video_2d_1x1x8_100e_kinetics400_rgb.log.json)|
+|[tsn_r50_video_dense_1x1x8_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_video_dense_1x1x8_100e_kinetics400_rgb.py) |short-side 256|8| ResNet50| ImageNet | 70.40 | 89.12 |x|x|x|21553| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_dense_1x1x8_100e_kinetics400_rgb/tsn_r50_video_dense_1x1x8_100e_kinetics400_rgb_20200703-0f19175f.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_dense_1x1x8_100e_kinetics400_rgb/tsn_r50_video_2d_1x1x8_dense_100e_kinetics400_rgb.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_dense_1x1x8_100e_kinetics400_rgb/tsn_r50_video_2d_1x1x8_dense_100e_kinetics400_rgb.log.json)|
+
+Here, We use [1: 1] to indicate that we combine rgb and flow score with coefficients 1: 1 to get the two-stream prediction (without applying softmax).
+
+### Kinetics-400 Data Benchmark (8-gpus, ResNet50, ImageNet pretrain; 3 segments)
+
+In data benchmark, we compare:
+1. Different data preprocessing methods: (1) Resize video to 340x256, (2) Resize the short edge of video to 320px, (3) Resize the short edge of video to 256px;
+2. Different data augmentation methods: (1) MultiScaleCrop, (2) RandomResizedCrop;
+3. Different testing protocols: (1) 25 frames x 10 crops, (2) 25 frames x 3 crops.
+
+| config | resolution | training augmentation | testing protocol | top1 acc | top5 acc | ckpt | log | json |
+| :----------------------------------------------------------: | :------------: | :-------------------: | :--------------: | :------: | :------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [tsn_r50_multiscalecrop_340x256_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_340x256_1x1x3_100e_kinetics400_rgb.py) | 340x256 | MultiScaleCrop | 25x10 frames | 70.60 | 89.26 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/20200614_063526.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/20200614_063526.log.json) |
+| x | 340x256 | MultiScaleCrop | 25x3 frames | 70.52 | 89.39 | x | x | x |
+| [tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb.py) | 340x256 | RandomResizedCrop | 25x10 frames | 70.11 | 89.01 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb/tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb_20200725-88cb325a.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb/tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb_20200725.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb/tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb_20200725.json) |
+| x | 340x256 | RandomResizedCrop | 25x3 frames | 69.95 | 89.02 | x | x | x |
+| [tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb.py) | short-side 320 | MultiScaleCrop | 25x10 frames | 70.32 | 89.25 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb_20200725-9922802f.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb_20200725.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb_20200725.json) |
+| x | short-side 320 | MultiScaleCrop | 25x3 frames | 70.54 | 89.39 | x | x | x |
+| [tsn_r50_randomresizedcrop_320p_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_320p_1x1x3_100e_kinetics400_rgb.py) | short-side 320 | RandomResizedCrop | 25x10 frames | 70.44 | 89.23 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_320p_1x1x3_100e_kinetics400_rgb_20200702-cc665e2a.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_f3_kinetics400_shortedge_70.9_89.5.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x3_100e_kinetics400_rgb/tsn_r50_f3_kinetics400_shortedge_70.9_89.5.log.json) |
+| x | short-side 320 | RandomResizedCrop | 25x3 frames | 70.91 | 89.51 | x | x | x |
+| [tsn_r50_multiscalecrop_256p_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_256p_1x1x3_100e_kinetics400_rgb.py) | short-side 256 | MultiScaleCrop | 25x10 frames | 70.42 | 89.03 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_1x1x3_100e_kinetics400_rgb/tsn_r50_256p_1x1x3_100e_kinetics400_rgb_20200725-22592236.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_1x1x3_100e_kinetics400_rgb/20200725_031325.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_1x1x3_100e_kinetics400_rgb/20200725_031325.log.json)|
+| x | short-side 256 | MultiScaleCrop | 25x3 frames | 70.79 | 89.42 | x | x | x |
+| [tsn_r50_randomresizedcrop_256p_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_256p_1x1x3_100e_kinetics400_rgb.py) | short-side 256 | RandomResizedCrop | 25x10 frames | 69.80 | 89.06 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_randomresize_1x1x3_100e_kinetics400_rgb/tsn_r50_256p_randomresize_1x1x3_100e_kinetics400_rgb_20200817-ae7963ca.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_randomresize_1x1x3_100e_kinetics400_rgb/20200815_172601.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_randomresize_1x1x3_100e_kinetics400_rgb/20200815_172601.log.json)|
+| x | short-side 256 | RandomResizedCrop | 25x3 frames | 70.48 | 89.89 | x | x | x |
+
+### Kinetics-400 OmniSource Experiments
+
+| config | resolution | backbone | pretrain | w. OmniSource | top1 acc | top5 acc | inference_time(video/s) | gpu_mem(M) | ckpt | log | json |
+| :----------------------------------------------------------: | :------------: | :------: | :-------: | :----------------: | :------: | :------: | :---------------------: | :--------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [tsn_r50_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py) | 340x256 | ResNet50 | ImageNet | :x: | 70.6 | 89.3 | 4.3 (25x10 frames) | 8344 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/20200614_063526.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/20200614_063526.log.json) |
+| x | 340x256 | ResNet50 | ImageNet | :heavy_check_mark: | 73.6 | 91.0 | x | 8344 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/omni/tsn_imagenet_pretrained_r50_omni_1x1x3_kinetics400_rgb_20200926-54192355.pth) | x | x |
+| x | short-side 320 | ResNet50 | IG-1B [1] | :x: | 73.1 | 90.4 | x | 8344 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/omni/tsn_1G1B_pretrained_r50_without_omni_1x1x3_kinetics400_rgb_20200926-c133dd49.pth) | x | x |
+| x | short-side 320 | ResNet50 | IG-1B [1] | :heavy_check_mark: | 75.7 | 91.9 | x | 8344 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/omni/tsn_1G1B_pretrained_r50_omni_1x1x3_kinetics400_rgb_20200926-2863fed0.pth) | x | x |
+
+[1] We obtain the pre-trained model from [torch-hub](https://pytorch.org/hub/facebookresearch_semi-supervised-ImageNet1K-models_resnext/), the pretrain model we used is `resnet50_swsl`
+
+### Kinetics-600
+
+| config | resolution | gpus | backbone | pretrain | top1 acc | top5 acc | inference_time(video/s) | gpu_mem(M) | ckpt | log | json |
+| :----------------------------------------------------------- | :------------: | :--: | :------: | :------: | :------: | :------: | :---------------------: | :--------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [tsn_r50_video_1x1x8_100e_kinetics600_rgb](/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics600_rgb.py) | short-side 256 | 8x2 | ResNet50 | ImageNet | 74.8 | 92.3 | 11.1 (25x3 frames) | 8344 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics600_rgb/tsn_r50_video_1x1x8_100e_kinetics600_rgb_20201015-4db3c461.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics600_rgb/tsn_r50_video_1x1x8_100e_kinetics600_rgb_20201015.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics600_rgb/tsn_r50_video_1x1x8_100e_kinetics600_rgb_20201015.json) |
+
+### Kinetics-700
+
+| config | resolution | gpus | backbone | pretrain | top1 acc | top5 acc | inference_time(video/s) | gpu_mem(M) | ckpt | log | json |
+| :----------------------------------------------------------- | :------------: | :--: | :------: | :------: | :------: | :------: | :---------------------: | :--------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [tsn_r50_video_1x1x8_100e_kinetics700_rgb](/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics700_rgb.py) | short-side 256 | 8x2 | ResNet50 | ImageNet | 61.7 | 83.6 | 11.1 (25x3 frames) | 8344 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics700_rgb/tsn_r50_video_1x1x8_100e_kinetics700_rgb_20201015-e381a6c7.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics700_rgb/tsn_r50_video_1x1x8_100e_kinetics700_rgb_20201015.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics700_rgb/tsn_r50_video_1x1x8_100e_kinetics700_rgb_20201015.json) |
+
+### Something-Something V1
+
+|config|resolution | gpus| backbone |pretrain| top1 acc| top5 acc | reference top1 acc | reference top5 acc | gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsn_r50_1x1x8_50e_sthv1_rgb](/configs/recognition/tsn/tsn_r50_1x1x8_50e_sthv1_rgb.py)|height 100 |8| ResNet50 | ImageNet|18.55 |44.80 |[17.53](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[44.29](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)| 10978 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_sthv1_rgb/tsn_r50_1x1x8_50e_sthv1_rgb_20200618-061b9195.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_sthv1_rgb/tsn_sthv1.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_sthv1_rgb/tsn_r50_f8_sthv1_18.1_45.0.log.json)|
+|[tsn_r50_1x1x16_50e_sthv1_rgb](/configs/recognition/tsn/tsn_r50_1x1x16_50e_sthv1_rgb.py)| height 100 |8| ResNet50| ImageNet |15.77 |39.85 |[13.33](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[35.58](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)| 5691 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x16_50e_sthv1_rgb/tsn_r50_1x1x16_50e_sthv1_rgb_20200614-7e2fe4f1.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x16_50e_sthv1_rgb/20200614_211932.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x16_50e_sthv1_rgb/20200614_211932.log.json)|
+
+### Something-Something V2
+
+|config |resolution| gpus| backbone| pretrain | top1 acc| top5 acc | reference top1 acc | reference top5 acc | gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsn_r50_1x1x8_50e_sthv2_rgb](/configs/recognition/tsn/tsn_r50_1x1x8_50e_sthv2_rgb.py)|height 240 |8| ResNet50| ImageNet |32.97 |63.62 |[30.56](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[58.49](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)| 10966 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_sthv2_rgb/tsn_r50_1x1x8_50e_sthv2_rgb_20200915-f3b381a5.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_sthv2_rgb/20200915_114139.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x8_50e_sthv2_rgb/20200915_114139.log.json)|
+|[tsn_r50_1x1x16_50e_sthv2_rgb](/configs/recognition/tsn/tsn_r50_1x1x16_50e_sthv2_rgb.py)| height 240 |8|ResNet50| ImageNet |27.21 |55.84 |[21.91](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|[46.87](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd#training)|8337| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x16_50e_sthv2_rgb/tsn_r50_1x1x16_50e_sthv2_rgb_20200917-80bc3611.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x16_50e_sthv2_rgb/20200917_105855.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x16_50e_sthv2_rgb/20200917_105855.log.json)|
+
+### Moments in Time
+
+|config |resolution| gpus| backbone | pretrain | top1 acc| top5 acc | gpu_mem(M)| ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsn_r50_1x1x6_100e_mit_rgb](/configs/recognition/tsn/tsn_r50_1x1x6_100e_mit_rgb.py)|short-side 256 |8x2| ResNet50| ImageNet |26.84|51.6| 8339| [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x6_100e_mit_rgb/tsn_r50_1x1x6_100e_mit_rgb_20200618-d512ab1b.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x6_100e_mit_rgb/tsn_mit.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x6_100e_mit_rgb/tsn_r50_f6_mit_26.8_51.6.log.json)|
+
+### Multi-Moments in Time
+
+|config | resolution|gpus| backbone | pretrain | mAP| gpu_mem(M) | ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsn_r101_1x1x5_50e_mmit_rgb](/configs/recognition/tsn/tsn_r101_1x1x5_50e_mmit_rgb.py)|short-side 256 |8x2| ResNet101| ImageNet |61.09| 10467 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r101_1x1x5_50e_mmit_rgb/tsn_r101_1x1x5_50e_mmit_rgb_20200618-642f450d.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r101_1x1x5_50e_mmit_rgb/tsn_mmit.log)| [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r101_1x1x5_50e_mmit_rgb/tsn_r101_f6_mmit_61.1.log.json)|
+
+### ActivityNet v1.3
+
+| config | resolution | gpus | backbone | pretrain | top1 acc | top5 acc | gpu_mem(M) | ckpt | log | json |
+| :----------------------------------------------------------- | :--------: | :--: | :------: | :---------: | :------: | :------: | :--------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [tsn_r50_320p_1x1x8_50e_activitynet_video_rgb](/configs/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb.py) | 340x256 | 8x1 | ResNet50 | Kinetics400 | 73.97 | 93.46 | 5692 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb_20200804-9e15687e.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb_20200804.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb_20200804.json) |
+| [tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb](/configs/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb.py) | 340x256 | 8x1 | ResNet50 | Kinetics400 | 76.07 | 94.10 | 5692 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb_20200804-f7bcbf34.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb_20200804.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb_20200804.json) |
+| [tsn_r50_320p_1x1x8_150e_activitynet_video_flow](/configs/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_video_flow.py) | 340x256 | 8x2 | ResNet50 | Kinetics400 | 58.70 | 84.72 | 5780 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_video_flow/tsn_r50_320p_1x1x8_150e_activitynet_video_flow_20200804-13313f52.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_video_flow/tsn_r50_320p_1x1x8_150e_activitynet_video_flow_20200804.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_video_flow/tsn_r50_320p_1x1x8_150e_activitynet_video_flow_20200804.json) |
+| [tsn_r50_320p_1x1x8_150e_activitynet_clip_flow](/configs/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow.py) | 340x256 | 8x2 | ResNet50 | Kinetics400 | 59.51 | 82.69 | 5780 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow_20200804-8622cf38.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow_20200804.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow_20200804.json) |
+
+### HVU
+
+| config[1] | tag category | resolution | gpus | backbone | pretrain | mAP | HATNet[2] | HATNet-multi[2] | ckpt | log | json |
+| :----------------------------------------------------------: | :----------: | :------------: | :--: | :------: | :------: | :--: | :-------: | :-------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [tsn_r18_1x1x8_100e_hvu_action_rgb](/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_action_rgb.py) | action | short-side 256 | 8x2 | ResNet18 | ImageNet | 57.5 | 51.8 | 53.5 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/action/tsn_r18_1x1x8_100e_hvu_action_rgb_20201027-011b282b.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/action/tsn_r18_1x1x8_100e_hvu_action_rgb_20201027.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/action/tsn_r18_1x1x8_100e_hvu_action_rgb_20201027.json) |
+| [tsn_r18_1x1x8_100e_hvu_scene_rgb](/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_scene_rgb.py) | scene | short-side 256 | 8 | ResNet18 | ImageNet | 55.2 | 55.8 | 57.2 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/scene/tsn_r18_1x1x8_100e_hvu_scene_rgb_20201027-00e5748d.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/scene/tsn_r18_1x1x8_100e_hvu_scene_rgb_20201027.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/scene/tsn_r18_1x1x8_100e_hvu_scene_rgb_20201027.json) |
+| [tsn_r18_1x1x8_100e_hvu_object_rgb](/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_object_rgb.py) | object | short-side 256 | 8 | ResNet18 | ImageNet | 45.7 | 34.2 | 35.1 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/object/tsn_r18_1x1x8_100e_hvu_object_rgb_20201102-24a22f30.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/object/tsn_r18_1x1x8_100e_hvu_object_rgb_20201027.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/object/tsn_r18_1x1x8_100e_hvu_object_rgb_20201027.json) |
+| [tsn_r18_1x1x8_100e_hvu_event_rgb](/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_event_rgb.py) | event | short-side 256 | 8 | ResNet18 | ImageNet | 63.7 | 38.5 | 39.8 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/event/tsn_r18_1x1x8_100e_hvu_event_rgb_20201027-dea8cd71.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/event/tsn_r18_1x1x8_100e_hvu_event_rgb_20201027.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/event/tsn_r18_1x1x8_100e_hvu_event_rgb_20201027.json) |
+| [tsn_r18_1x1x8_100e_hvu_concept_rgb](/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_concept_rgb.py) | concept | short-side 256 | 8 | ResNet18 | ImageNet | 47.5 | 26.1 | 27.3 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/concept/tsn_r18_1x1x8_100e_hvu_concept_rgb_20201027-fc1dd8e3.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/concept/tsn_r18_1x1x8_100e_hvu_concept_rgb_20201027.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/concept/tsn_r18_1x1x8_100e_hvu_concept_rgb_20201027.json) |
+| [tsn_r18_1x1x8_100e_hvu_attribute_rgb](/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_attribute_rgb.py) | attribute | short-side 256 | 8 | ResNet18 | ImageNet | 46.1 | 33.6 | 34.9 | [ckpt](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/attribute/tsn_r18_1x1x8_100e_hvu_attribute_rgb_20201027-0b3b49d2.pth) | [log](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/attribute/tsn_r18_1x1x8_100e_hvu_attribute_rgb_20201027.log) | [json](https://download.openmmlab.com/mmaction/recognition/tsn/hvu/attribute/tsn_r18_1x1x8_100e_hvu_attribute_rgb_20201027.json) |
+| - | Overall | short-side 256 | - | ResNet18 | ImageNet | 52.6 | 40.0 | 41.3 | - | - | - |
+
+[1] For simplicity, we train a specific model for each tag category as the baselines for HVU.
+
+[2] The performance of HATNet and HATNet-multi are from the paper [Large Scale Holistic Video Understanding](https://pages.iai.uni-bonn.de/gall_juergen/download/HVU_eccv20.pdf). The proposed HATNet is a 2 branch Convolution Network (one 2D branch, one 3D branch) and share the same backbone(ResNet18) with us. The inputs of HATNet are 16 or 32 frames long video clips (which is much larger than us), while the input resolution is coarser (112 instead of 224). HATNet is trained on each individual task (each tag category) while HATNet-multi is trained on multiple tasks. Since there is no released codes or models for the HATNet, we just include the performance reported by the original paper.
+
+Notes:
+
+1. The **gpus** indicates the number of gpu we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+ According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+ e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+ not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+3. The values in columns named after "reference" are the results got by training on the original repo, using the same model settings.
+
+For more details on data preparation, you can refer to
+
+* [preparing_ucf101](/tools/data/ucf101/README.md)
+* [preparing_kinetics](/tools/data/kinetics/README.md)
+* [preparing_sthv1](/tools/data/sthv1/README.md)
+* [preparing_sthv2](/tools/data/sthv2/README.md)
+* [preparing_mit](/tools/data/mit/README.md)
+* [preparing_mmit](/tools/data/mmit/README.md)
+* [preparing_hvu](/tools/data/hvu/README.md)
+* [preparing_hmdb51](/tools/data/hmdb51/README.md)
+
+## Train
+
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train TSN model on Kinetics-400 dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py \
+ --work-dir work_dirs/tsn_r50_1x1x3_100e_kinetics400_rgb \
+ --validate --seed 0 --deterministic
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test TSN model on Kinetics-400 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_256p_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_256p_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..90cd4c6
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_256p_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,116 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train_256p'
+data_root_val = 'data/kinetics400/rawframes_val_256p'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes_256p.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes_256p.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_256p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/tsn_r50_multiscalecrop_256p_1x1x3'
+ '_100e_kinetics400_rgb/')
+load_from = None
+resume_from = None
+workflow = [('train', 5)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..caacec2
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_320p_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,116 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train_320p'
+data_root_val = 'data/kinetics400/rawframes_val_320p'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes_320p.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/tsn_r50_multiscalecrop_320p_1x1x3'
+ '_100e_kinetics400_rgb/')
+load_from = None
+resume_from = None
+workflow = [('train', 5)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_340x256_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_340x256_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..ef56b33
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_multiscalecrop_340x256_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,120 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/tsn_r50_multiscalecrop_340x256_1x1x3'
+ '_100e_kinetics400_rgb/')
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_256p_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_256p_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..d25e9f0
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_256p_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,115 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train_256p'
+data_root_val = 'data/kinetics400/rawframes_val_256p'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes_256p.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes_256p.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_256p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/tsn_r50_randomresizedcrop_256p_1x1x3'
+ '_100e_kinetics400_rgb/')
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_320p_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_320p_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..f1e9009
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_320p_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,115 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train_320p'
+data_root_val = 'data/kinetics400/rawframes_val_320p'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes_320p.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/tsn_r50_randomresizedcrop_320p_1x1x3'
+ '_100e_kinetics400_rgb/')
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..e2497ea
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_randomresizedcrop_340x256_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,111 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/tsn_r50_randomresizedcrop_340x256_1x1x3'
+ '_100e_kinetics400_rgb')
+load_from = None
+resume_from = None
+workflow = [('train', 5)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_256p_1x1x25_10crop_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_256p_1x1x25_10crop_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..56da32e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_256p_1x1x25_10crop_100e_kinetics400_rgb.py
@@ -0,0 +1,49 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root_val = 'data/kinetics400/rawframes_val_256p'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_256p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+dist_params = dict(backend='nccl')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_256p_1x1x25_3crop_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_256p_1x1x25_3crop_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..3822444
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_256p_1x1x25_3crop_100e_kinetics400_rgb.py
@@ -0,0 +1,49 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root_val = 'data/kinetics400/rawframes_val_256p'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_256p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+dist_params = dict(backend='nccl')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_320p_1x1x25_10crop_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_320p_1x1x25_10crop_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..8c1b928
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_320p_1x1x25_10crop_100e_kinetics400_rgb.py
@@ -0,0 +1,49 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root_val = 'data/kinetics400/rawframes_val_320p'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+dist_params = dict(backend='nccl')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_320p_1x1x25_3crop_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_320p_1x1x25_3crop_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..77c5d87
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_320p_1x1x25_3crop_100e_kinetics400_rgb.py
@@ -0,0 +1,49 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root_val = 'data/kinetics400/rawframes_val_320p'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+dist_params = dict(backend='nccl')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_340x256_1x1x25_10crop_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_340x256_1x1x25_10crop_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..a6c2819
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_340x256_1x1x25_10crop_100e_kinetics400_rgb.py
@@ -0,0 +1,49 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+dist_params = dict(backend='nccl')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_340x256_1x1x25_3crop_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_340x256_1x1x25_3crop_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..5f600f2
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/data_benchmark/tsn_r50_test_340x256_1x1x25_3crop_100e_kinetics400_rgb.py
@@ -0,0 +1,49 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ workers_per_gpu=4,
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+dist_params = dict(backend='nccl')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_action_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_action_rgb.py
new file mode 100644
index 0000000..09463f2
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_action_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+category_nums = dict(
+ action=739, attribute=117, concept=291, event=69, object=1678, scene=248)
+target_cate = 'action'
+
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet18',
+ depth=18,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=category_nums[target_cate],
+ in_channels=512,
+ spatial_type='avg',
+ multi_class=True,
+ consensus=dict(type='AvgConsensus', dim=1),
+ loss_cls=dict(type='BCELossWithLogits', loss_weight=333.),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/hvu/rawframes_train'
+data_root_val = 'data/hvu/rawframes_val'
+ann_file_train = f'data/hvu/hvu_{target_cate}_train.json'
+ann_file_val = f'data/hvu/hvu_{target_cate}_val.json'
+ann_file_test = f'data/hvu/hvu_{target_cate}_val.json'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(interval=2, metrics=['mean_average_precision'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = f'./work_dirs/tsn_r18_1x1x8_100e_hvu_{target_cate}_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_attribute_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_attribute_rgb.py
new file mode 100644
index 0000000..d6135ba
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_attribute_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+category_nums = dict(
+ action=739, attribute=117, concept=291, event=69, object=1678, scene=248)
+target_cate = 'attribute'
+
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet18',
+ depth=18,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=category_nums[target_cate],
+ in_channels=512,
+ spatial_type='avg',
+ multi_class=True,
+ consensus=dict(type='AvgConsensus', dim=1),
+ loss_cls=dict(type='BCELossWithLogits', loss_weight=333.),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/hvu/rawframes_train'
+data_root_val = 'data/hvu/rawframes_val'
+ann_file_train = f'data/hvu/hvu_{target_cate}_train.json'
+ann_file_val = f'data/hvu/hvu_{target_cate}_val.json'
+ann_file_test = f'data/hvu/hvu_{target_cate}_val.json'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(interval=2, metrics=['mean_average_precision'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = f'./work_dirs/tsn_r18_1x1x8_100e_hvu_{target_cate}_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_concept_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_concept_rgb.py
new file mode 100644
index 0000000..fe90f5c
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_concept_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+category_nums = dict(
+ action=739, attribute=117, concept=291, event=69, object=1678, scene=248)
+target_cate = 'object'
+
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet18',
+ depth=18,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=category_nums[target_cate],
+ in_channels=512,
+ spatial_type='avg',
+ multi_class=True,
+ consensus=dict(type='AvgConsensus', dim=1),
+ loss_cls=dict(type='BCELossWithLogits', loss_weight=333.),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/hvu/rawframes_train'
+data_root_val = 'data/hvu/rawframes_val'
+ann_file_train = f'data/hvu/hvu_{target_cate}_train.json'
+ann_file_val = f'data/hvu/hvu_{target_cate}_val.json'
+ann_file_test = f'data/hvu/hvu_{target_cate}_val.json'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(interval=2, metrics=['mean_average_precision'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = f'./work_dirs/tsn_r18_1x1x8_100e_hvu_{target_cate}_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_event_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_event_rgb.py
new file mode 100644
index 0000000..d4b011b
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_event_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+category_nums = dict(
+ action=739, attribute=117, concept=291, event=69, object=1678, scene=248)
+target_cate = 'event'
+
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet18',
+ depth=18,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=category_nums[target_cate],
+ in_channels=512,
+ spatial_type='avg',
+ multi_class=True,
+ consensus=dict(type='AvgConsensus', dim=1),
+ loss_cls=dict(type='BCELossWithLogits', loss_weight=333.),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/hvu/rawframes_train'
+data_root_val = 'data/hvu/rawframes_val'
+ann_file_train = f'data/hvu/hvu_{target_cate}_train.json'
+ann_file_val = f'data/hvu/hvu_{target_cate}_val.json'
+ann_file_test = f'data/hvu/hvu_{target_cate}_val.json'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(interval=2, metrics=['mean_average_precision'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = f'./work_dirs/tsn_r18_1x1x8_100e_hvu_{target_cate}_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_object_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_object_rgb.py
new file mode 100644
index 0000000..fe90f5c
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_object_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+category_nums = dict(
+ action=739, attribute=117, concept=291, event=69, object=1678, scene=248)
+target_cate = 'object'
+
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet18',
+ depth=18,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=category_nums[target_cate],
+ in_channels=512,
+ spatial_type='avg',
+ multi_class=True,
+ consensus=dict(type='AvgConsensus', dim=1),
+ loss_cls=dict(type='BCELossWithLogits', loss_weight=333.),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/hvu/rawframes_train'
+data_root_val = 'data/hvu/rawframes_val'
+ann_file_train = f'data/hvu/hvu_{target_cate}_train.json'
+ann_file_val = f'data/hvu/hvu_{target_cate}_val.json'
+ann_file_test = f'data/hvu/hvu_{target_cate}_val.json'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(interval=2, metrics=['mean_average_precision'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = f'./work_dirs/tsn_r18_1x1x8_100e_hvu_{target_cate}_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_scene_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_scene_rgb.py
new file mode 100644
index 0000000..7026649
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/hvu/tsn_r18_1x1x8_100e_hvu_scene_rgb.py
@@ -0,0 +1,126 @@
+# model settings
+category_nums = dict(
+ action=739, attribute=117, concept=291, event=69, object=1678, scene=248)
+target_cate = 'scene'
+
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet18',
+ depth=18,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=category_nums[target_cate],
+ in_channels=512,
+ spatial_type='avg',
+ multi_class=True,
+ consensus=dict(type='AvgConsensus', dim=1),
+ loss_cls=dict(type='BCELossWithLogits', loss_weight=333.),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/hvu/rawframes_train'
+data_root_val = 'data/hvu/rawframes_val'
+ann_file_train = f'data/hvu/hvu_{target_cate}_train.json'
+ann_file_val = f'data/hvu/hvu_{target_cate}_val.json'
+ann_file_test = f'data/hvu/hvu_{target_cate}_val.json'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ multi_class=True,
+ num_classes=category_nums[target_cate],
+ filename_tmpl='img_{:05d}.jpg'))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(interval=2, metrics=['mean_average_precision'])
+log_config = dict(
+ interval=20, hooks=[
+ dict(type='TextLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = f'./work_dirs/tsn_r18_1x1x8_100e_hvu_{target_cate}_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_fp16_r50_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_fp16_r50_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..75e180a
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_fp16_r50_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,121 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# fp16 settings
+fp16 = dict()
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_fp16_r50_1x1x3_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r101_1x1x5_50e_mmit_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r101_1x1x5_50e_mmit_rgb.py
new file mode 100644
index 0000000..a71f0e2
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r101_1x1x5_50e_mmit_rgb.py
@@ -0,0 +1,134 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet101',
+ depth=101,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=313,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ loss_cls=dict(type='BCELossWithLogits', loss_weight=160.0),
+ dropout_ratio=0.5,
+ init_std=0.01,
+ multi_class=True,
+ label_smooth_eps=0))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/mmit/rawframes/training'
+data_root_val = '/data/mmit/rawframes/validation/'
+ann_file_train = 'data/mmit/mmit_train_list_rawframes.txt'
+ann_file_val = 'data/mmit/mmit_val_list_rawframes.txt'
+ann_file_test = 'data/mmit/mmit_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=5),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=5,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=5,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='MultiGroupCrop', crop_size=256, groups=1),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline,
+ multi_class=True,
+ num_classes=313),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ multi_class=True,
+ num_classes=313),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ multi_class=True,
+ num_classes=313))
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='TSMOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.01, # this lr is used for 8 gpus
+ momentum=0.9,
+ weight_decay=0.0001,
+)
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=5)
+evaluation = dict(interval=5, metrics=['mmit_mean_average_precision'])
+# yapf:disable
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r101_1x1x5_50e_mmit_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x16_50e_sthv1_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x16_50e_sthv1_rgb.py
new file mode 100644
index 0000000..f885928
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x16_50e_sthv1_rgb.py
@@ -0,0 +1,122 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_cfg=dict(type='SyncBN', requires_grad=True),
+ norm_eval=True),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv1/rawframes'
+data_root_val = 'data/sthv1/rawframes'
+ann_file_train = 'data/sthv1/sthv1_train_list_rawframes.txt'
+ann_file_val = 'data/sthv1/sthv1_val_list_rawframes.txt'
+ann_file_test = 'data/sthv1/sthv1_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=16),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=4,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='{:05}.jpg',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0005) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x16_50e_sthv1_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x16_50e_sthv2_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x16_50e_sthv2_rgb.py
new file mode 100644
index 0000000..e0eecd1
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x16_50e_sthv2_rgb.py
@@ -0,0 +1,107 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv2/rawframes'
+data_root_val = 'data/sthv2/rawframes'
+ann_file_train = 'data/sthv2/sthv2_train_list_rawframes.txt'
+ann_file_val = 'data/sthv2/sthv2_val_list_rawframes.txt'
+ann_file_test = 'data/sthv2/sthv2_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=16),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=16),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=16),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=4,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.005, momentum=0.9,
+ weight_decay=0.0005) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x16_50e_sthv2_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..a25d5db
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,119 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x3_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x3_75e_ucf101_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x3_75e_ucf101_rgb.py
new file mode 100644
index 0000000..e012139
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x3_75e_ucf101_rgb.py
@@ -0,0 +1,115 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=101,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.001))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/ucf101/rawframes/'
+data_root_val = 'data/ucf101/rawframes/'
+split = 1 # official train/test splits. valid numbers: 1, 2, 3
+ann_file_train = f'data/ucf101/ucf101_train_split_{split}_rawframes.txt'
+ann_file_val = f'data/ucf101/ucf101_val_split_{split}_rawframes.txt'
+ann_file_test = f'data/ucf101/ucf101_val_split_{split}_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.00128, momentum=0.9,
+ weight_decay=0.0005) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[])
+total_epochs = 75
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = f'./work_dirs/tsn_r50_1x1x3_75e_ucf101_split_{split}_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x6_100e_mit_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x6_100e_mit_rgb.py
new file mode 100644
index 0000000..1ef66f4
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x6_100e_mit_rgb.py
@@ -0,0 +1,117 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=339,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/mit/rawframes/training'
+data_root_val = '/data/mit/rawframes/validation/'
+ann_file_train = 'data/mit/mit_train_list_rawframes.txt'
+ann_file_val = 'data/mit/mit_val_list_rawframes.txt'
+ann_file_test = 'data/mit/mit_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=6),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=6,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=6,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.005, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x6_100e_mit_rgb'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_imagenet_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_imagenet_rgb.py
new file mode 100644
index 0000000..6263652
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_imagenet_rgb.py
@@ -0,0 +1,107 @@
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=51,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips=None)
+dataset_type = 'RawframeDataset'
+data_root = 'data/hmdb51/rawframes'
+data_root_val = 'data/hmdb51/rawframes'
+ann_file_train = 'data/hmdb51/hmdb51_train_split_1_rawframes.txt'
+ann_file_val = 'data/hmdb51/hmdb51_val_split_1_rawframes.txt'
+ann_file_test = 'data/hmdb51/hmdb51_val_split_1_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+optimizer = dict(type='SGD', lr=0.025, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=1, metrics=['top_k_accuracy', 'mean_class_accuracy'], topk=(1, 5))
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook')
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x8_50e_hmdb51_imagenet_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+gpu_ids = range(0, 1)
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb.py
new file mode 100644
index 0000000..1364145
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb.py
@@ -0,0 +1,107 @@
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=51,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips=None)
+dataset_type = 'RawframeDataset'
+data_root = 'data/hmdb51/rawframes'
+data_root_val = 'data/hmdb51/rawframes'
+ann_file_train = 'data/hmdb51/hmdb51_train_split_1_rawframes.txt'
+ann_file_val = 'data/hmdb51/hmdb51_val_split_1_rawframes.txt'
+ann_file_test = 'data/hmdb51/hmdb51_val_split_1_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+optimizer = dict(type='SGD', lr=0.025, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=1, metrics=['top_k_accuracy', 'mean_class_accuracy'], topk=(1, 5))
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook')
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb/'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_256p_1x1x8_100e_kinetics400_rgb/tsn_r50_256p_1x1x8_100e_kinetics400_rgb_20200817-883baf16.pth' # noqa: E501
+resume_from = None
+workflow = [('train', 1)]
+gpu_ids = range(0, 1)
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_mit_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_mit_rgb.py
new file mode 100644
index 0000000..8c9fe3f
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_mit_rgb.py
@@ -0,0 +1,105 @@
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=51,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+train_cfg = None
+test_cfg = dict(average_clips=None)
+dataset_type = 'RawframeDataset'
+data_root = 'data/hmdb51/rawframes'
+data_root_val = 'data/hmdb51/rawframes'
+ann_file_train = 'data/hmdb51/hmdb51_train_split_1_rawframes.txt'
+ann_file_val = 'data/hmdb51/hmdb51_val_split_1_rawframes.txt'
+ann_file_test = 'data/hmdb51/hmdb51_val_split_1_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+optimizer = dict(type='SGD', lr=0.025, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=1, metrics=['top_k_accuracy', 'mean_class_accuracy'], topk=(1, 5))
+log_config = dict(
+ interval=5,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook')
+ ])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x8_50e_hmdb51_mit_rgb/'
+load_from = 'https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x6_100e_mit_rgb/tsn_r50_1x1x6_100e_mit_rgb_20200618-d512ab1b.pth' # noqa: E501
+resume_from = None
+workflow = [('train', 1)]
+gpu_ids = range(0, 1)
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_sthv1_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_sthv1_rgb.py
new file mode 100644
index 0000000..5a4b5e3
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_sthv1_rgb.py
@@ -0,0 +1,123 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_cfg=dict(type='SyncBN', requires_grad=True),
+ norm_eval=True),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv1/rawframes'
+data_root_val = 'data/sthv1/rawframes'
+ann_file_train = 'data/sthv1/sthv1_train_list_rawframes.txt'
+ann_file_val = 'data/sthv1/sthv1_val_list_rawframes.txt'
+ann_file_test = 'data/sthv1/sthv1_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='{:05}.jpg',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='{:05}.jpg',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.02, momentum=0.9,
+ weight_decay=0.0005) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x8_50e_sthv1_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_sthv2_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_sthv2_rgb.py
new file mode 100644
index 0000000..1e938b6
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_1x1x8_50e_sthv2_rgb.py
@@ -0,0 +1,109 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=174,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/sthv2/rawframes'
+data_root_val = 'data/sthv2/rawframes'
+ann_file_train = 'data/sthv2/sthv2_train_list_rawframes.txt'
+ann_file_val = 'data/sthv2/sthv2_val_list_rawframes.txt'
+ann_file_test = 'data/sthv2/sthv2_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.02, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x8_50e_sthv2_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..aa6d60f
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,114 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train_320p'
+data_root_val = 'data/kinetics400/rawframes_val_320p'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes_320p.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_320p_1x1x3_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x3_110e_kinetics400_flow.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x3_110e_kinetics400_flow.py
new file mode 100644
index 0000000..f040bad
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x3_110e_kinetics400_flow.py
@@ -0,0 +1,120 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train_320p'
+data_root_val = 'data/kinetics400/rawframes_val_320p'
+ann_file_train = 'data/kinetics400/kinetics_flow_train_list.txt'
+ann_file_val = 'data/kinetics400/kinetics_flow_val_list.txt'
+ann_file_test = 'data/kinetics400/kinetics_flow_val_list.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=5, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=5,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=5,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='{}_{:05d}.jpg',
+ modality='Flow',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='{}_{:05d}.jpg',
+ modality='Flow',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='{}_{:05d}.jpg',
+ modality='Flow',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.005, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[70, 100])
+total_epochs = 110
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_320p_1x1x3_110e_kinetics400_flow/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..22b1e64
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_100e_kinetics400_rgb.py
@@ -0,0 +1,114 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train_320p'
+data_root_val = 'data/kinetics400/rawframes_val_320p'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes_320p.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes_320p.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.00375, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_320p_1x1x8_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_110e_kinetics400_flow.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_110e_kinetics400_flow.py
new file mode 100644
index 0000000..7108130
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_110e_kinetics400_flow.py
@@ -0,0 +1,120 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train_320p'
+data_root_val = 'data/kinetics400/rawframes_val_320p'
+ann_file_train = 'data/kinetics400/kinetics_flow_train_list.txt'
+ann_file_val = 'data/kinetics400/kinetics_flow_val_list.txt'
+ann_file_test = 'data/kinetics400/kinetics_flow_val_list.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=5, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=5,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=5,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='{}_{:05d}.jpg',
+ modality='Flow',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='{}_{:05d}.jpg',
+ modality='Flow',
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='{}_{:05d}.jpg',
+ modality='Flow',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.001875, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[70, 100])
+total_epochs = 110
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_320p_1x1x8_110e_kinetics400_flow/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow.py
new file mode 100644
index 0000000..645202e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow.py
@@ -0,0 +1,125 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='modelzoo/tsn_r50_320p_1x1x8_110e_kinetics400_flow.pth',
+ depth=50,
+ in_channels=10,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=200,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.8,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/ActivityNet/flow'
+data_root_val = 'data/ActivityNet/flow'
+ann_file_train = 'data/ActivityNet/anet_train_clip.txt'
+ann_file_val = 'data/ActivityNet/anet_val_clip.txt'
+ann_file_test = 'data/ActivityNet/anet_val_clip.txt'
+img_norm_cfg = dict(mean=[128, 128], std=[128, 128], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=5, frame_interval=1, num_clips=8),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=5,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=5,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='flow_{}_{:05d}.jpg',
+ with_offset=True,
+ modality='Flow',
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='flow_{}_{:05d}.jpg',
+ with_offset=True,
+ modality='Flow',
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='flow_{}_{:05d}.jpg',
+ with_offset=True,
+ modality='Flow',
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001)
+# this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[60, 120])
+total_epochs = 150
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_320p_1x1x8_150e_activitynet_clip_flow/'
+load_from = None
+resume_from = None
+workflow = [('train', 5)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_video_flow.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_video_flow.py
new file mode 100644
index 0000000..2911a12
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_150e_activitynet_video_flow.py
@@ -0,0 +1,125 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='modelzoo/tsn_r50_320p_1x1x8_110e_kinetics400_flow.pth',
+ depth=50,
+ in_channels=10,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=200,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.8,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/ActivityNet/flow'
+data_root_val = 'data/ActivityNet/flow'
+ann_file_train = 'data/ActivityNet/anet_train_video.txt'
+ann_file_val = 'data/ActivityNet/anet_val_video.txt'
+ann_file_test = 'data/ActivityNet/anet_val_clip.txt'
+img_norm_cfg = dict(mean=[128, 128], std=[128, 128], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=5, frame_interval=1, num_clips=8),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=5,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=5,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW_Flow'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ filename_tmpl='flow_{}_{:05d}.jpg',
+ with_offset=True,
+ modality='Flow',
+ start_index=0,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ filename_tmpl='flow_{}_{:05d}.jpg',
+ with_offset=True,
+ modality='Flow',
+ start_index=0,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ filename_tmpl='flow_{}_{:05d}.jpg',
+ with_offset=True,
+ modality='Flow',
+ start_index=0,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001)
+# this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[60, 120])
+total_epochs = 150
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_320p_1x1x8_150e_activitynet_video_flow/'
+load_from = None
+resume_from = None
+workflow = [('train', 5)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb.py
new file mode 100644
index 0000000..f97d70e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb.py
@@ -0,0 +1,122 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='modelzoo/tsn_r50_320p_1x1x8_100e_kinetics400_rgb.pth',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=200,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.8,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/ActivityNet/rgb'
+data_root_val = 'data/ActivityNet/rgb'
+ann_file_train = 'data/ActivityNet/anet_train_clip.txt'
+ann_file_val = 'data/ActivityNet/anet_val_clip.txt'
+ann_file_test = 'data/ActivityNet/anet_val_clip.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline,
+ with_offset=True,
+ start_index=0,
+ filename_tmpl='image_{:05d}.jpg'),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ with_offset=True,
+ start_index=0,
+ filename_tmpl='image_{:05d}.jpg'),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ with_offset=True,
+ start_index=0,
+ filename_tmpl='image_{:05d}.jpg'))
+# optimizer
+optimizer = dict(type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001)
+# this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_320p_1x1x8_50e_activitynet_clip_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 5)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb.py
new file mode 100644
index 0000000..c2c016e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb.py
@@ -0,0 +1,122 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='modelzoo/tsn_r50_320p_1x1x8_100e_kinetics400_rgb.pth',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=200,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.8,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/ActivityNet/rgb'
+data_root_val = 'data/ActivityNet/rgb'
+ann_file_train = 'data/ActivityNet/anet_train_video.txt'
+ann_file_val = 'data/ActivityNet/anet_val_video.txt'
+ann_file_test = 'data/ActivityNet/anet_val_clip.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='FrameSelector'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline,
+ with_offset=True,
+ start_index=0,
+ filename_tmpl='image_{:05d}.jpg'),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline,
+ with_offset=True,
+ start_index=0,
+ filename_tmpl='image_{:05d}.jpg'),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline,
+ with_offset=True,
+ start_index=0,
+ filename_tmpl='image_{:05d}.jpg'))
+# optimizer
+optimizer = dict(type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001)
+# this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[20, 40])
+total_epochs = 50
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_320p_1x1x8_50e_activitynet_video_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 5)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_dense_1x1x5_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_dense_1x1x5_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..580c126
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_dense_1x1x5_100e_kinetics400_rgb.py
@@ -0,0 +1,121 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.5,
+ init_std=0.001))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DenseSampleFrames', clip_len=1, frame_interval=1, num_clips=5),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=16,
+ workers_per_gpu=4,
+ val_dataloader=dict(videos_per_gpu=4),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.03, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=20, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=2, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_dense_1x1x5_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_dense_1x1x8_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_dense_1x1x8_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..349e423
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_dense_1x1x8_100e_kinetics400_rgb.py
@@ -0,0 +1,120 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DenseSampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1,
+ num_fixed_crops=13),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.005, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_dense_1x1x8_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_inference_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_inference_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..6941d2a
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_inference_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,46 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'RawframeDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..43305ed
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics400_rgb.py
@@ -0,0 +1,121 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics400/videos_train'
+data_root_val = 'data/kinetics400/videos_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_videos.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_videos.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_video_1x1x8_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics600_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics600_rgb.py
new file mode 100644
index 0000000..4bd634f
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics600_rgb.py
@@ -0,0 +1,116 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=600,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics600/videos_train'
+data_root_val = 'data/kinetics600/videos_val'
+ann_file_train = 'data/kinetics600/kinetics600_train_list_videos.txt'
+ann_file_val = 'data/kinetics600/kinetics600_val_list_videos.txt'
+ann_file_test = 'data/kinetics600/kinetics600_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.00375, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x3_100e_kinetics600_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics700_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics700_rgb.py
new file mode 100644
index 0000000..9155224
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics700_rgb.py
@@ -0,0 +1,116 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=700,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics700/videos_train'
+data_root_val = 'data/kinetics700/videos_val'
+ann_file_train = 'data/kinetics700/kinetics700_train_list_videos.txt'
+ann_file_val = 'data/kinetics700/kinetics700_val_list_videos.txt'
+ann_file_test = 'data/kinetics700/kinetics700_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=12,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.00375, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_1x1x3_100e_kinetics700_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..476c2d5
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,116 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics400/videos_train'
+data_root_val = 'data/kinetics400/videos_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_videos.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_videos.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='DecordDecode'),
+ dict(type='RandomResizedCrop'),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_video_1x1x3_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_dense_1x1x8_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_dense_1x1x8_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..5870693
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_dense_1x1x8_100e_kinetics400_rgb.py
@@ -0,0 +1,122 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips=None)
+# dataset settings
+dataset_type = 'VideoDataset'
+data_root = 'data/kinetics400/videos_train'
+data_root_val = 'data/kinetics400/videos_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_videos.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_videos.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_videos.txt'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='DecordInit'),
+ dict(type='DenseSampleFrames', clip_len=1, frame_interval=1, num_clips=8),
+ dict(type='DecordDecode'),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(type='DecordInit'),
+ dict(
+ type='DenseSampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=8,
+ test_mode=True),
+ dict(type='DecordDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ val_dataloader=dict(videos_per_gpu=1),
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.01, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 100
+checkpoint_config = dict(interval=1)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_video_dense_1x1x8_100e_kinetics400_rgb/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py
new file mode 100644
index 0000000..b22ca57
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/tsn/tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py
@@ -0,0 +1,46 @@
+# model settings
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+# model training and testing settings
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'VideoDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+test_pipeline = [
+ dict(type='OpenCVInit', num_threads=1),
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='OpenCVDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=None,
+ data_prefix=None,
+ pipeline=test_pipeline))
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/x3d/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition/x3d/README.md
new file mode 100644
index 0000000..a3a0942
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/x3d/README.md
@@ -0,0 +1,45 @@
+# X3D
+
+## Introduction
+```
+@misc{feichtenhofer2020x3d,
+ title={X3D: Expanding Architectures for Efficient Video Recognition},
+ author={Christoph Feichtenhofer},
+ year={2020},
+ eprint={2004.04730},
+ archivePrefix={arXiv},
+ primaryClass={cs.CV}
+}
+```
+
+## Model Zoo
+
+### Kinetics-400
+
+|config | resolution | backbone | top1 10-view | top1 30-view | reference top1 10-view | reference top1 30-view | ckpt |
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[x3d_s_13x6x1_facebook_kinetics400_rgb](/configs/recognition/x3d/x3d_s_13x6x1_facebook_kinetics400_rgb.py)|short-side 320| X3D_S | 72.7 | 73.2 | 73.1 [[SlowFast](https://github.com/facebookresearch/SlowFast/blob/master/MODEL_ZOO.md)] | 73.5 [[SlowFast](https://github.com/facebookresearch/SlowFast/blob/master/MODEL_ZOO.md)] | [ckpt](https://download.openmmlab.com/mmaction/recognition/x3d/facebook/x3d_s_facebook_13x6x1_kinetics400_rgb_20201027-623825a0.pth)[1] |
+|[x3d_m_16x5x1_facebook_kinetics400_rgb](/configs/recognition/x3d/x3d_m_16x5x1_facebook_kinetics400_rgb)|short-side 320| X3D_M | 75.0 | 75.6 | 75.1 [[SlowFast](https://github.com/facebookresearch/SlowFast/blob/master/MODEL_ZOO.md)] | 76.2 [[SlowFast](https://github.com/facebookresearch/SlowFast/blob/master/MODEL_ZOO.md)] | [ckpt](https://download.openmmlab.com/mmaction/recognition/x3d/facebook/x3d_m_facebook_16x5x1_kinetics400_rgb_20201027-3f42382a.pth)[1] |
+
+[1] The models are ported from the repo [SlowFast](https://github.com/facebookresearch/SlowFast/) and tested on our data. Currently, we only support the testing of X3D models, training will be available soon.
+
+Notes:
+
+3. The values in columns named after "reference" are the results got by testing the checkpoint released on the original repo and codes, using the same dataset with ours.
+
+For more details on data preparation, you can refer to Kinetics400 in [Data Preparation](/docs/data_preparation.md).
+
+## Test
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test X3D model on Kinetics-400 dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/recognition/x3d/x3d_s_13x6x1_facebook_kinetics400_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json --average-clips prob
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/x3d/x3d_m_16x5x1_facebook_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/x3d/x3d_m_16x5x1_facebook_kinetics400_rgb.py
new file mode 100644
index 0000000..ebc06c7
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/x3d/x3d_m_16x5x1_facebook_kinetics400_rgb.py
@@ -0,0 +1,45 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(type='X3D', gamma_w=1, gamma_b=2.25, gamma_d=2.2),
+ cls_head=dict(
+ type='X3DHead',
+ in_channels=432,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ fc1_bias=False))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[114.75, 114.75, 114.75], std=[57.38, 57.38, 57.38], to_bgr=False)
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=16,
+ frame_interval=5,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=256),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+
+dist_params = dict(backend='nccl')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition/x3d/x3d_s_13x6x1_facebook_kinetics400_rgb.py b/Downstream/Open-Set-Action-Recognition/configs/recognition/x3d/x3d_s_13x6x1_facebook_kinetics400_rgb.py
new file mode 100644
index 0000000..e378316
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition/x3d/x3d_s_13x6x1_facebook_kinetics400_rgb.py
@@ -0,0 +1,45 @@
+# model settings
+model = dict(
+ type='Recognizer3D',
+ backbone=dict(type='X3D', gamma_w=1, gamma_b=2.25, gamma_d=2.2),
+ cls_head=dict(
+ type='X3DHead',
+ in_channels=432,
+ num_classes=400,
+ spatial_type='avg',
+ dropout_ratio=0.5,
+ fc1_bias=False))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+img_norm_cfg = dict(
+ mean=[114.75, 114.75, 114.75], std=[57.38, 57.38, 57.38], to_bgr=False)
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=13,
+ frame_interval=6,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 192)),
+ dict(type='CenterCrop', crop_size=192),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+data = dict(
+ videos_per_gpu=1,
+ workers_per_gpu=2,
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+
+dist_params = dict(backend='nccl')
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/audioonly/audioonly_r50_64x1x1_100e_kinetics400_audio_feature.py b/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/audioonly/audioonly_r50_64x1x1_100e_kinetics400_audio_feature.py
new file mode 100644
index 0000000..a223876
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/audioonly/audioonly_r50_64x1x1_100e_kinetics400_audio_feature.py
@@ -0,0 +1,102 @@
+# model settings
+model = dict(
+ type='AudioRecognizer',
+ backbone=dict(
+ type='ResNetAudio',
+ depth=50,
+ pretrained=None,
+ in_channels=1,
+ norm_eval=False),
+ cls_head=dict(
+ type='AudioTSNHead',
+ num_classes=400,
+ in_channels=1024,
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'AudioFeatureDataset'
+data_root = 'data/kinetics400/audio_feature_train'
+data_root_val = 'data/kinetics400/audio_feature_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_audio_feature.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_audio_feature.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_audio_feature.txt'
+train_pipeline = [
+ dict(type='LoadAudioFeature'),
+ dict(type='SampleFrames', clip_len=64, frame_interval=1, num_clips=1),
+ dict(type='AudioFeatureSelector'),
+ dict(type='FormatAudioShape', input_format='NCTF'),
+ dict(type='Collect', keys=['audios', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['audios'])
+]
+val_pipeline = [
+ dict(type='LoadAudioFeature'),
+ dict(
+ type='SampleFrames',
+ clip_len=64,
+ frame_interval=1,
+ num_clips=1,
+ test_mode=True),
+ dict(type='AudioFeatureSelector'),
+ dict(type='FormatAudioShape', input_format='NCTF'),
+ dict(type='Collect', keys=['audios', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['audios'])
+]
+test_pipeline = [
+ dict(type='LoadAudioFeature'),
+ dict(
+ type='SampleFrames',
+ clip_len=64,
+ frame_interval=1,
+ num_clips=10,
+ test_mode=True),
+ dict(type='AudioFeatureSelector'),
+ dict(type='FormatAudioShape', input_format='NCTF'),
+ dict(type='Collect', keys=['audios', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['audios'])
+]
+data = dict(
+ videos_per_gpu=160,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=2.0, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=1,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = ('./work_dirs/' +
+ 'audioonly_r50_64x1x1_100e_kinetics400_audio_feature/')
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/resnet/README.md b/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/resnet/README.md
new file mode 100644
index 0000000..90d03aa
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/resnet/README.md
@@ -0,0 +1,73 @@
+# ResNet for Audio
+
+## Introduction
+```
+@article{xiao2020audiovisual,
+ title={Audiovisual SlowFast Networks for Video Recognition},
+ author={Xiao, Fanyi and Lee, Yong Jae and Grauman, Kristen and Malik, Jitendra and Feichtenhofer, Christoph},
+ journal={arXiv preprint arXiv:2001.08740},
+ year={2020}
+}
+```
+
+## Model Zoo
+
+### Kinetics-400
+
+|config | n_fft | gpus | backbone |pretrain| top1 acc/delta| top5 acc/delta | inference_time(video/s) | gpu_mem(M)| ckpt | log| json|
+|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+|[tsn_r18_64x1x1_100e_kinetics400_audio_feature](/configs/recognition_audio/resnet/tsn_r18_64x1x1_100e_kinetics400_audio_feature.py)|1024|8| ResNet18 | None |19.7|35.75|x|1897|[ckpt](https://download.openmmlab.com/mmaction/recognition/audio_recognition/tsn_r18_64x1x1_100e_kinetics400_audio_feature/tsn_r18_64x1x1_100e_kinetics400_audio_feature_20201012-bf34df6c.pth)|[log](https://download.openmmlab.com/mmaction/recognition/audio_recognition/tsn_r18_64x1x1_100e_kinetics400_audio_feature/20201010_144630.log)|[json](https://download.openmmlab.com/mmaction/recognition/audio_recognition/tsn_r18_64x1x1_100e_kinetics400_audio_feature/20201010_144630.log.json)|
+|[tsn_r18_64x1x1_100e_kinetics400_audio_feature](/configs/recognition_audio/resnet/tsn_r18_64x1x1_100e_kinetics400_audio_feature.py) + [tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb](/configs/recognition/tsn/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb.py)|1024|8| ResNet(18+50) | None |71.50(+0.39)|90.18(+0.14)|x|x|x|x|x|
+
+Notes:
+
+1. The **gpus** indicates the number of gpus we used to get the checkpoint. It is noteworthy that the configs we provide are used for 8 gpus as default.
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you may set the learning rate proportional to the batch size if you use different GPUs or videos per GPU,
+e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+2. The **inference_time** is got by this [benchmark script](/tools/analysis/benchmark.py), where we use the sampling frames strategy of the test setting and only care about the model inference time,
+not including the IO time and pre-processing time. For each setting, we use 1 gpu and set batch size (videos per gpu) to 1 to calculate the inference time.
+3. The values in columns named after "reference" are the results got by training on the original repo, using the same model settings.
+
+For more details on data preparation, you can refer to Kinetics400 in [Data Preparation](/docs/data_preparation.md).
+
+## Train
+
+You can use the following command to train a model.
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+Example: train ResNet model on Kinetics-400 audio dataset in a deterministic option with periodic validation.
+```shell
+python tools/train.py configs/audio_recognition/tsn_r50_64x1x1_100e_kinetics400_audio_feature.py \
+ --work-dir work_dirs/tsn_r50_64x1x1_100e_kinetics400_audio_feature \
+ --validate --seed 0 --deterministic
+```
+
+For more details, you can refer to **Training setting** part in [getting_started](/docs/getting_started.md#training-setting).
+
+## Test
+
+You can use the following command to test a model.
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+Example: test ResNet model on Kinetics-400 audio dataset and dump the result to a json file.
+```shell
+python tools/test.py configs/audio_recognition/tsn_r50_64x1x1_100e_kinetics400_audio_feature.py \
+ checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
+ --out result.json
+```
+
+For more details, you can refer to **Test a dataset** part in [getting_started](/docs/getting_started.md#test-a-dataset).
+
+## Fusion
+For multi-modality fusion, you can use the simple [script](/tools/analysis/report_accuracy.py), the standard usage is:
+
+```shell
+python tools/analysis/report_accuracy.py --scores ${AUDIO_RESULT_PKL} ${VISUAL_RESULT_PKL} --datalist data/kinetics400/kinetics400_val_list_rawframes.txt --coefficient 1 1
+```
+
++ AUDIO_RESULT_PKL: The saved output file of `tools/test.py` by the argument `--out`.
++ VISUAL_RESULT_PKL: The saved output file of `tools/test.py` by the argument `--out`.
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/resnet/tsn_r18_64x1x1_100e_kinetics400_audio_feature.py b/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/resnet/tsn_r18_64x1x1_100e_kinetics400_audio_feature.py
new file mode 100644
index 0000000..4c85dbc
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/resnet/tsn_r18_64x1x1_100e_kinetics400_audio_feature.py
@@ -0,0 +1,96 @@
+# model settings
+model = dict(
+ type='AudioRecognizer',
+ backbone=dict(type='ResNet', depth=18, in_channels=1, norm_eval=False),
+ cls_head=dict(
+ type='AudioTSNHead',
+ num_classes=400,
+ in_channels=512,
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'AudioFeatureDataset'
+data_root = 'data/kinetics400/audio_feature_train'
+data_root_val = 'data/kinetics400/audio_feature_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_audio_feature.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_audio_feature.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_audio_feature.txt'
+train_pipeline = [
+ dict(type='LoadAudioFeature'),
+ dict(type='SampleFrames', clip_len=64, frame_interval=1, num_clips=1),
+ dict(type='AudioFeatureSelector'),
+ dict(type='FormatAudioShape', input_format='NCTF'),
+ dict(type='Collect', keys=['audios', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['audios'])
+]
+val_pipeline = [
+ dict(type='LoadAudioFeature'),
+ dict(
+ type='SampleFrames',
+ clip_len=64,
+ frame_interval=1,
+ num_clips=1,
+ test_mode=True),
+ dict(type='AudioFeatureSelector'),
+ dict(type='FormatAudioShape', input_format='NCTF'),
+ dict(type='Collect', keys=['audios', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['audios'])
+]
+test_pipeline = [
+ dict(type='LoadAudioFeature'),
+ dict(
+ type='SampleFrames',
+ clip_len=64,
+ frame_interval=1,
+ num_clips=1,
+ test_mode=True),
+ dict(type='AudioFeatureSelector'),
+ dict(type='FormatAudioShape', input_format='NCTF'),
+ dict(type='Collect', keys=['audios', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['audios'])
+]
+data = dict(
+ videos_per_gpu=320,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.1, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r18_64x1x1_100e_kinetics400_audio_feature/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/resnet/tsn_r50_64x1x1_100e_kinetics400_audio.py b/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/resnet/tsn_r50_64x1x1_100e_kinetics400_audio.py
new file mode 100644
index 0000000..4086e42
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/configs/recognition_audio/resnet/tsn_r50_64x1x1_100e_kinetics400_audio.py
@@ -0,0 +1,102 @@
+# model settings
+model = dict(
+ type='AudioRecognizer',
+ backbone=dict(type='ResNet', depth=50, in_channels=1, norm_eval=False),
+ cls_head=dict(
+ type='AudioTSNHead',
+ num_classes=400,
+ in_channels=2048,
+ dropout_ratio=0.5,
+ init_std=0.01))
+# model training and testing settings
+train_cfg = None
+test_cfg = dict(average_clips='prob')
+# dataset settings
+dataset_type = 'AudioDataset'
+data_root = 'data/kinetics400/audios'
+data_root_val = 'data/kinetics400/audios'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_audio.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_audio.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_audio.txt'
+train_pipeline = [
+ dict(type='AudioDecodeInit'),
+ dict(type='SampleFrames', clip_len=64, frame_interval=1, num_clips=1),
+ dict(type='AudioDecode'),
+ dict(type='AudioAmplify', ratio=1.5),
+ dict(type='MelLogSpectrogram'),
+ dict(type='FormatAudioShape', input_format='NCTF'),
+ dict(type='Collect', keys=['audios', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['audios'])
+]
+val_pipeline = [
+ dict(type='AudioDecodeInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=64,
+ frame_interval=1,
+ num_clips=1,
+ test_mode=True),
+ dict(type='AudioDecode'),
+ dict(type='AudioAmplify', ratio=1.5),
+ dict(type='MelLogSpectrogram'),
+ dict(type='FormatAudioShape', input_format='NCTF'),
+ dict(type='Collect', keys=['audios', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['audios'])
+]
+test_pipeline = [
+ dict(type='AudioDecodeInit'),
+ dict(
+ type='SampleFrames',
+ clip_len=64,
+ frame_interval=1,
+ num_clips=1,
+ test_mode=True),
+ dict(type='AudioDecodeInit'),
+ dict(type='AudioAmplify', ratio=1.5),
+ dict(type='MelLogSpectrogram'),
+ dict(type='FormatAudioShape', input_format='NCTF'),
+ dict(type='Collect', keys=['audios', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['audios'])
+]
+data = dict(
+ videos_per_gpu=320,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(
+ type='SGD', lr=0.1, momentum=0.9,
+ weight_decay=0.0001) # this lr is used for 8 gpus
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='CosineAnnealing', min_lr=0)
+total_epochs = 100
+checkpoint_config = dict(interval=5)
+evaluation = dict(
+ interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])
+log_config = dict(
+ interval=20,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook'),
+ ])
+# runtime settings
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/tsn_r50_64x1x1_100e_kinetics400_audio/'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/Downstream/Open-Set-Action-Recognition/docker/Dockerfile b/Downstream/Open-Set-Action-Recognition/docker/Dockerfile
new file mode 100644
index 0000000..f746272
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docker/Dockerfile
@@ -0,0 +1,25 @@
+ARG PYTORCH="1.6.0"
+ARG CUDA="10.1"
+ARG CUDNN="7"
+
+FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel
+
+ENV TORCH_CUDA_ARCH_LIST="6.0 6.1 7.0+PTX"
+ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"
+ENV CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"
+
+RUN apt-get update && apt-get install -y git ninja-build libglib2.0-0 libsm6 libxrender-dev libxext6 ffmpeg \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
+
+# Install mmcv-full
+RUN pip install mmcv-full==latest+torch1.6.0+cu101 -f https://download.openmmlab.com/mmcv/dist/index.html
+
+# Install MMAction2
+RUN conda clean --all
+RUN git clone https://github.com/open-mmlab/mmaction2.git /mmaction2
+WORKDIR /mmaction2
+RUN mkdir -p /mmaction2/data
+ENV FORCE_CUDA="1"
+RUN pip install cython --no-cache-dir
+RUN pip install --no-cache-dir -e .
diff --git a/Downstream/Open-Set-Action-Recognition/docs/Makefile b/Downstream/Open-Set-Action-Recognition/docs/Makefile
new file mode 100644
index 0000000..d4bb2cb
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/Makefile
@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS ?=
+SPHINXBUILD ?= sphinx-build
+SOURCEDIR = .
+BUILDDIR = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+ @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+ @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
diff --git a/Downstream/Open-Set-Action-Recognition/docs/api.rst b/Downstream/Open-Set-Action-Recognition/docs/api.rst
new file mode 100644
index 0000000..364003b
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/api.rst
@@ -0,0 +1,104 @@
+API Reference
+=============
+
+mmaction.apis
+-------------
+.. automodule:: mmaction.apis
+ :members:
+
+mmaction.core
+-------------
+
+optimizer
+^^^^^^^^^
+.. automodule:: mmaction.core.optimizer
+ :members:
+
+evaluation
+^^^^^^^^^^
+.. automodule:: mmaction.core.evaluation
+ :members:
+
+lr
+^^
+.. automodule:: mmaction.core.lr
+ :members:
+
+mmaction.localization
+---------------------
+
+localization
+^^^^^^^^^^^^
+.. automodule:: mmaction.localization
+ :members:
+
+mmaction.models
+---------------
+
+models
+^^^^^^
+.. automodule:: mmaction.models
+ :members:
+
+recognizers
+^^^^^^^^^^^
+.. automodule:: mmaction.models.recognizers
+ :members:
+
+localizers
+^^^^^^^^^^
+.. automodule:: mmaction.models.localizers
+ :members:
+
+common
+^^^^^^
+.. automodule:: mmaction.models.common
+ :members:
+
+backbones
+^^^^^^^^^
+.. automodule:: mmaction.models.backbones
+ :members:
+
+heads
+^^^^^
+.. automodule:: mmaction.models.heads
+ :members:
+
+necks
+^^^^^
+.. automodule:: mmaction.models.necks
+ :members:
+
+losses
+^^^^^^
+.. automodule:: mmaction.models.losses
+ :members:
+
+mmaction.datasets
+-----------------
+
+datasets
+^^^^^^^^
+.. automodule:: mmaction.datasets
+ :members:
+
+pipelines
+^^^^^^^^^
+.. automodule:: mmaction.datasets.pipelines
+ :members:
+
+samplers
+^^^^^^^^
+.. automodule:: mmaction.datasets.samplers
+ :members:
+
+mmaction.utils
+--------------
+.. automodule:: mmaction.utils
+ :members:
+
+mmaction.localization
+---------------------
+.. automodule:: mmaction.localization
+ :members:
diff --git a/Downstream/Open-Set-Action-Recognition/docs/benchmark.md b/Downstream/Open-Set-Action-Recognition/docs/benchmark.md
new file mode 100644
index 0000000..5eb81ec
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/benchmark.md
@@ -0,0 +1,137 @@
+# Benchmark
+
+We compare our results with some popular frameworks and official releases in terms of speed.
+
+## Settings
+### Hardware
+
+- 8 NVIDIA Tesla V100 (32G) GPUs
+- Intel(R) Xeon(R) Gold 6146 CPU @ 3.20GHz
+
+### Software Environment
+
+- Python 3.7
+- PyTorch 1.4
+- CUDA 10.1
+- CUDNN 7.6.03
+- NCCL 2.4.08
+
+### Metrics
+The time we measured is the average training time for an iteration, including data processing and model training.
+The training speed is measure with s/iter. The lower, the better. Note that we skip the first 50 iter times as they may contain the device warmup time.
+
+### Comparison Rules
+
+Here we compare our MMAction2 repo with other video understanding toolboxes in the same data and model settings
+by the training time per iteration. Here, we use
+- commit id [7f3490d](https://github.com/open-mmlab/mmaction/tree/7f3490d3db6a67fe7b87bfef238b757403b670e3)(1/5/2020) of MMAction
+- commit id [8d53d6f](https://github.com/mit-han-lab/temporal-shift-module/tree/8d53d6fda40bea2f1b37a6095279c4b454d672bd)(5/5/2020) of Temporal-Shift-Module
+- commit id [8299c98](https://github.com/facebookresearch/SlowFast/tree/8299c9862f83a067fa7114ce98120ae1568a83ec)(7/7/2020) of PySlowFast
+- commit id [f13707f](https://github.com/wzmsltw/BSN-boundary-sensitive-network/tree/f13707fbc362486e93178c39f9c4d398afe2cb2f)(12/12/2018) of BSN(boundary sensitive network)
+- commit id [45d0514](https://github.com/JJBOY/BMN-Boundary-Matching-Network/tree/45d05146822b85ca672b65f3d030509583d0135a)(17/10/2019) of BMN(boundary matching network)
+
+To ensure the fairness of the comparison, the comparison experiments were conducted under the same hardware environment and using the same dataset. The rawframe dataset we used is generated by the [data preparation tools](/tools/data/kinetics/README.md), the video dataset we used is a special version of resized video cache called '256p dense-encoded video', featuring a faster decoding speed which is generated by the scripts [here](/tools/data/resize_video.py). Significant improvement can be observed when comparing with normal 256p videos as shown in the table below, especially when the sampling is sparse(like [TSN](/configs/recognition/tsn/tsn_r50_video_320p_1x1x3_100e_kinetics400_rgb.py)).
+
+For each model setting, we kept the same data preprocessing methods to make sure the same feature input.
+In addition, we also used Memcached, a distributed cached system, to load the data for the same IO time except for fair comparisons with Pyslowfast which uses raw videos directly from disk by default.
+
+We provide the training log based on which we calculate the average iter time, with the actual setting logged inside, feel free to verify it and fire an issue if something does not make sense.
+
+## Main Results
+### Recognizers
+
+| Model |input| io backend | batch size x gpus | MMAction2 (s/iter) | GPU mem(GB) | MMAction (s/iter)| GPU mem(GB) | Temporal-Shift-Module (s/iter) | GPU mem(GB) | PySlowFast (s/iter)| GPU mem(GB) |
+| :--- | :---------------:|:---------------:| :---------------:| :---------------: | :--------------------: | :----------------------------: | :-----------------: |:-----------------: |:-----------------: |:-----------------: |:-----------------: |
+| [TSN](/configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py)| 256p rawframes |Memcached| 32x8|**[0.32](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/tsn_256p_rawframes_memcahed_32x8.zip)** | 8.1 |[0.38](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction/tsn_256p_rawframes_memcached_32x8.zip)|8.1| [0.42](https://download.openmmlab.com/mmaction/benchmark/recognition/temporal_shift_module/tsn_256p_rawframes_memcached_32x8.zip)|10.5 | x |x |
+| [TSN](/configs/recognition/tsn/tsn_r50_video_1x1x3_100e_kinetics400_rgb.py)| 256p videos |Disk| 32x8|**[1.42](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/tsn_256p_videos_disk_32x8.zip)** | 8.1 | x |x |x| x | TODO |TODO|
+| [TSN](/configs/recognition/tsn/tsn_r50_video_1x1x3_100e_kinetics400_rgb.py)| 256p dense-encoded video |Disk| 32x8|**[0.61](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/tsn_256p_fast_videos_disk_32x8.zip)**|8.1 | x |x| x |x| TODO |TODO|
+|[I3D heavy](/configs/recognition/i3d/i3d_r50_video_heavy_8x8x1_100e_kinetics400_rgb.py)|256p videos|Disk |8x8| **[0.34](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/i3d_heavy_256p_videos_disk_8x8.zip)** |4.6|x |x| x |x| [0.44](https://download.openmmlab.com/mmaction/benchmark/recognition/pyslowfast/pysf_i3d_r50_8x8_video.log) |4.6|
+|[I3D heavy](/configs/recognition/i3d/i3d_r50_video_heavy_8x8x1_100e_kinetics400_rgb.py)|256p dense-encoded video|Disk |8x8| **[0.35](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/i3d_heavy_256p_fast_videos_disk_8x8.zip)**| 4.6 | x | x | x | x | [0.36](https://download.openmmlab.com/mmaction/benchmark/recognition/pyslowfast/pysf_i3d_r50_8x8_fast_video.log) |4.6|
+| [I3D](/configs/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb.py)|256p rawframes|Memcached|8x8| **[0.43](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/i3d_256p_rawframes_memcahed_8x8.zip)**|5.0 | [0.56](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction/i3d_256p_rawframes_memcached_8x8.zip)|5.0| x |x| x |x|
+| [TSM](/configs/recognition/tsm/tsm_r50_1x1x8_50e_kinetics400_rgb.py) |256p rawframes|Memcached| 8x8|**[0.31](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/tsm_256p_rawframes_memcahed_8x8.zip)** |6.9| x |x| [0.41](https://download.openmmlab.com/mmaction/benchmark/recognition/temporal_shift_module/tsm_256p_rawframes_memcached_8x8.zip) |9.1| x|x |
+| [Slowonly](/configs/recognition/slowonly/slowonly_r50_video_4x16x1_256e_kinetics400_rgb.py)|256p videos|Disk|8x8 | **[0.32](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/slowonly_256p_videos_disk_8x8.zip)** |3.1| TODO|TODO | x|x | [0.34](https://download.openmmlab.com/mmaction/benchmark/recognition/pyslowfast/pysf_slowonly_r50_4x16_video.log)|3.4 |
+| [Slowonly](/configs/recognition/slowonly/slowonly_r50_video_4x16x1_256e_kinetics400_rgb.py)|256p dense-encoded video|Disk|8x8 | **[0.25](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/slowonly_256p_fast_videos_disk_8x8.zip)** |3.1| TODO |TODO| x |x| [0.28](https://download.openmmlab.com/mmaction/benchmark/recognition/pyslowfast/pysf_slowonly_r50_4x16_fast_video.log) |3.4|
+| [Slowfast](/configs/recognition/slowfast/slowfast_r50_video_4x16x1_256e_kinetics400_rgb.py)|256p videos|Disk|8x8 | **[0.69](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/slowfast_256p_videos_disk_8x8.zip)**|6.1 | x |x| x |x| [1.04](https://download.openmmlab.com/mmaction/benchmark/recognition/pyslowfast/pysf_slowfast_r50_4x16_video.log)|7.0 |
+| [Slowfast](/configs/recognition/slowfast/slowfast_r50_video_4x16x1_256e_kinetics400_rgb.py)|256p dense-encoded video|Disk|8x8 | **[0.68](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/slowfast_256p_fast_videos_disk_8x8.zip)** |6.1| x|x | x |x| [0.96](https://download.openmmlab.com/mmaction/benchmark/recognition/pyslowfast/pysf_slowfast_r50_4x16_fast_video.log)|7.0 |
+| [R(2+1)D](/configs/recognition/r2plus1d/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb.py)|256p videos |Disk| 8x8|**[0.45](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/r2plus1d_256p_videos_disk_8x8.zip)**|5.1 | x | x | x | x | x | x |
+| [R(2+1)D](/configs/recognition/r2plus1d/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb.py)|256p dense-encoded video |Disk| 8x8|**[0.44](https://download.openmmlab.com/mmaction/benchmark/recognition/mmaction2/r2plus1d_256p_fast_videos_disk_8x8.zip)** |5.1| x|x | x |x| x |x|
+
+### Localizers
+
+| Model | MMAction2 (s/iter) | BSN(boundary sensitive network) (s/iter) |BMN(boundary matching network) (s/iter)|
+| :--- | :---------------: | :-------------------------------------: | :-------------------------------------: |
+| BSN ([TEM + PEM + PGM](/configs/localization/bsn)) | **0.074(TEM)+0.040(PEM)** | 0.101(TEM)+0.040(PEM) | x |
+| BMN ([bmn_400x100_2x8_9e_activitynet_feature](/configs/localization/bmn/bmn_400x100_2x8_9e_activitynet_feature.py)) | **3.27** | x | 3.30 |
+
+
+## Details of Comparison
+### TSN
++ **mmaction2**
+```shell
+# rawframes
+bash tools/slurm_train.sh ${PARTATION_NAME} benchmark_tsn configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py --work-dir work_dirs/benchmark_tsn_rawframes
+
+# videos
+bash tools/slurm_train.sh ${PARTATION_NAME} benchmark_tsn configs/recognition/tsn/tsn_r50_video_1x1x3_100e_kinetics400_rgb.py --work-dir work_dirs/benchmark_tsn_video
+```
+
++ **mmaction**
+```shell
+python -u tools/train_recognizer.py configs/TSN/tsn_kinetics400_2d_rgb_r50_seg3_f1s1.py
+```
+
++ **Temporal-Shift-Module**
+```shell
+python main.py kinetics RGB --arch resnet50 --num_segments 3 --gd 20 --lr 0.02 --wd 1e-4 --lr_steps 20 40 --epochs 1 --batch-size 256 -j 32 --dropout 0.5 --consensus_type=avg --eval-freq=10 --npb --print-freq 1
+```
+
+### I3D
++ **mmaction2**
+```shell
+# rawframes
+bash tools/slurm_train.sh ${PARTATION_NAME} benchmark_i3d configs/recognition/i3d/i3d_r50_32x2x1_100e_kinetics400_rgb.py --work-dir work_dirs/benchmark_i3d_rawframes
+
+# videos
+bash tools/slurm_train.sh ${PARTATION_NAME} benchmark_i3d configs/recognition/i3d/i3d_r50_video_heavy_8x8x1_100e_kinetics400_rgb.py --work-dir work_dirs/benchmark_i3d_video
+```
+
++ **mmaction**
+```shell
+python -u tools/train_recognizer.py configs/I3D_RGB/i3d_kinetics400_3d_rgb_r50_c3d_inflate3x1x1_seg1_f32s2.py
+```
+
++ **PySlowFast**
+```shell
+python tools/run_net.py --cfg configs/Kinetics/I3D_8x8_R50.yaml DATA.PATH_TO_DATA_DIR ${DATA_ROOT} NUM_GPUS 8 TRAIN.BATCH_SIZE 64 TRAIN.AUTO_RESUME False LOG_PERIOD 1 SOLVER.MAX_EPOCH 1 > pysf_i3d_r50_8x8_video.log
+```
+You may reproduce the result by writting a simple script to parse out the value of the field 'time_diff'.
+
+### SlowFast
++ **mmaction2**
+```shell
+bash tools/slurm_train.sh ${PARTATION_NAME} benchmark_slowfast configs/recognition/slowfast/slowfast_r50_video_4x16x1_256e_kinetics400_rgb.py --work-dir work_dirs/benchmark_slowfast_video
+```
+
++ **PySlowFast**
+```shell
+python tools/run_net.py --cfg configs/Kinetics/SLOWFAST_4x16_R50.yaml DATA.PATH_TO_DATA_DIR ${DATA_ROOT} NUM_GPUS 8 TRAIN.BATCH_SIZE 64 TRAIN.AUTO_RESUME False LOG_PERIOD 1 SOLVER.MAX_EPOCH 1 > pysf_slowfast_r50_4x16_video.log
+```
+You may reproduce the result by writting a simple script to parse out the value of the field 'time_diff'.
+
+### SlowOnly
++ **mmaction2**
+```shell
+bash tools/slurm_train.sh ${PARTATION_NAME} benchmark_slowonly configs/recognition/slowonly/slowonly_r50_video_4x16x1_256e_kinetics400_rgb.py --work-dir work_dirs/benchmark_slowonly_video
+```
+
++ **PySlowFast**
+```shell
+python tools/run_net.py --cfg configs/Kinetics/SLOW_4x16_R50.yaml DATA.PATH_TO_DATA_DIR ${DATA_ROOT} NUM_GPUS 8 TRAIN.BATCH_SIZE 64 TRAIN.AUTO_RESUME False LOG_PERIOD 1 SOLVER.MAX_EPOCH 1 > pysf_slowonly_r50_4x16_video.log
+```
+You may reproduce the result by writting a simple script to parse out the value of the field 'time_diff'.
+
+### R2plus1D
++ **mmaction2**
+```shell
+bash tools/slurm_train.sh ${PARTATION_NAME} benchmark_r2plus1d configs/recognition/r2plus1d/r2plus1d_r34_video_8x8x1_180e_kinetics400_rgb.py --work-dir work_dirs/benchmark_r2plus1d_video
+```
diff --git a/Downstream/Open-Set-Action-Recognition/docs/changelog.md b/Downstream/Open-Set-Action-Recognition/docs/changelog.md
new file mode 100644
index 0000000..c27a9fd
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/changelog.md
@@ -0,0 +1,198 @@
+## Changelog
+
+### 0.10.0 (31/12/2020)
+
+**Highlights**
+
+**New Features**
+
+**Improvements**
+- Add FAQ documents for easy troubleshooting. ([#413](https://github.com/open-mmlab/mmaction2/pull/413), [#420](https://github.com/open-mmlab/mmaction2/pull/420))
+
+
+**Bug and Typo Fixes**
+
+**ModelZoo**
+
+### 0.9.0 (30/11/2020)
+
+**Highlights**
+- Support GradCAM utils for recognizers
+- Support ResNet Audio model
+
+**New Features**
+- Automatically add modelzoo statistics to readthedocs ([#327](https://github.com/open-mmlab/mmaction2/pull/327))
+- Support GYM99 data preparation ([#331](https://github.com/open-mmlab/mmaction2/pull/331))
+- Add AudioOnly Pathway from AVSlowFast. ([#355](https://github.com/open-mmlab/mmaction2/pull/355))
+- Add GradCAM utils for recognizer ([#324](https://github.com/open-mmlab/mmaction2/pull/324))
+- Add print config script ([#345](https://github.com/open-mmlab/mmaction2/pull/345))
+- Add online motion vector decoder ([#291](https://github.com/open-mmlab/mmaction2/pull/291))
+
+**Improvements**
+- Support PyTorch 1.7 in CI ([#312](https://github.com/open-mmlab/mmaction2/pull/312))
+- Support to predict different labels in a long video ([#274](https://github.com/open-mmlab/mmaction2/pull/274))
+- Update docs bout test crops ([#359](https://github.com/open-mmlab/mmaction2/pull/359))
+- Polish code format using pylint manually ([#338](https://github.com/open-mmlab/mmaction2/pull/338))
+- Update unittest coverage ([#358](https://github.com/open-mmlab/mmaction2/pull/358), [#322](https://github.com/open-mmlab/mmaction2/pull/322), [#325](https://github.com/open-mmlab/mmaction2/pull/325))
+- Add random seed for building filelists ([#323](https://github.com/open-mmlab/mmaction2/pull/323))
+- Update colab tutorial ([#367](https://github.com/open-mmlab/mmaction2/pull/367))
+- set default batch_size of evaluation and testing to 1 ([#250](https://github.com/open-mmlab/mmaction2/pull/250))
+- Rename the preparation docs to `README.md` ([#388](https://github.com/open-mmlab/mmaction2/pull/388))
+- Move docs about demo to `demo/README.md` ([#329](https://github.com/open-mmlab/mmaction2/pull/329))
+- Remove redundant code in `tools/test.py` ([#310](https://github.com/open-mmlab/mmaction2/pull/310))
+- Automatically calculate number of test clips for Recognizer2D ([#359](https://github.com/open-mmlab/mmaction2/pull/359))
+
+**Bug and Typo Fixes**
+- Fix rename Kinetics classnames bug ([#384](https://github.com/open-mmlab/mmaction2/pull/384))
+- Fix a bug in BaseDataset when `data_prefix` is None ([#314](https://github.com/open-mmlab/mmaction2/pull/314))
+- Fix a bug about `tmp_folder` in `OpenCVInit` ([#357](https://github.com/open-mmlab/mmaction2/pull/357))
+- Fix `get_thread_id` when not using disk as backend ([#354](https://github.com/open-mmlab/mmaction2/pull/354), [#357](https://github.com/open-mmlab/mmaction2/pull/357))
+- Fix the bug of HVU object `num_classes` from 1679 to 1678 ([#307](https://github.com/open-mmlab/mmaction2/pull/307))
+- Fix typo in `export_model.md` ([#399](https://github.com/open-mmlab/mmaction2/pull/399))
+- Fix OmniSource training configs ([#321](https://github.com/open-mmlab/mmaction2/pull/321))
+- Fix Issue #306: Bug of SampleAVAFrames ([#317](https://github.com/open-mmlab/mmaction2/pull/317))
+
+**ModelZoo**
+- Add SlowOnly model for GYM99, both RGB and Flow ([#336](https://github.com/open-mmlab/mmaction2/pull/336))
+- Add auto modelzoo statistics in readthedocs ([#327](https://github.com/open-mmlab/mmaction2/pull/327))
+- Add TSN for HMDB51 pretrained on Kinetics400, Moments in Time and ImageNet. ([#372](https://github.com/open-mmlab/mmaction2/pull/372))
+
+### v0.8.0 (31/10/2020)
+
+**Highlights**
+- Support [OmniSource](https://arxiv.org/abs/2003.13042)
+- Support C3D
+- Support video recognition with audio modality
+- Support HVU
+- Support X3D
+
+**New Features**
+- Support AVA dataset preparation ([#266](https://github.com/open-mmlab/mmaction2/pull/266))
+- Support the training of video recognition dataset with multiple tag categories ([#235](https://github.com/open-mmlab/mmaction2/pull/235))
+- Support joint training with multiple training datasets of multiple formats, including images, untrimmed videos, etc. ([#242](https://github.com/open-mmlab/mmaction2/pull/242))
+- Support to specify a start epoch to conduct evaluation ([#216](https://github.com/open-mmlab/mmaction2/pull/216))
+- Implement X3D models, support testing with model weights converted from SlowFast ([#288](https://github.com/open-mmlab/mmaction2/pull/288))
+- Support specify a start epoch to conduct evaluation ([#216](https://github.com/open-mmlab/mmaction2/pull/216))
+
+**Improvements**
+- Set default values of 'average_clips' in each config file so that there is no need to set it explicitly during testing in most cases ([#232](https://github.com/open-mmlab/mmaction2/pull/232))
+- Extend HVU datatools to generate individual file list for each tag category ([#258](https://github.com/open-mmlab/mmaction2/pull/258))
+- Support data preparation for Kinetics-600 and Kinetics-700 ([#254](https://github.com/open-mmlab/mmaction2/pull/254))
+- Use `metric_dict` to replace hardcoded arguments in `evaluate` function ([#286](https://github.com/open-mmlab/mmaction2/pull/286))
+- Add `cfg-options` in arguments to override some settings in the used config for convenience ([#212](https://github.com/open-mmlab/mmaction2/pull/212))
+- Rename the old evaluating protocol `mean_average_precision` as `mmit_mean_average_precision` since it is only used on MMIT and is not the `mAP` we usually talk about. Add `mean_average_precision`, which is the real `mAP` ([#235](https://github.com/open-mmlab/mmaction2/pull/235))
+- Add accurate setting (Three crop * 2 clip) and report corresponding performance for TSM model ([#241](https://github.com/open-mmlab/mmaction2/pull/241))
+- Add citations in each preparing_dataset.md in `tools/data/dataset` ([#289](https://github.com/open-mmlab/mmaction2/pull/289))
+- Update the performance of audio-visual fusion on Kinetics-400 ([#281](https://github.com/open-mmlab/mmaction2/pull/281))
+- Support data preparation of OmniSource web datasets, including GoogleImage, InsImage, InsVideo and KineticsRawVideo ([#294](https://github.com/open-mmlab/mmaction2/pull/294))
+- Use `metric_options` dict to provide metric args in `evaluate` ([#286](https://github.com/open-mmlab/mmaction2/pull/286))
+
+**Bug Fixes**
+- Register `FrameSelector` in `PIPELINES` ([#268](https://github.com/open-mmlab/mmaction2/pull/268))
+- Fix the potential bug for default value in dataset_setting ([#245](https://github.com/open-mmlab/mmaction2/pull/245))
+- Fix multi-node dist test ([#292](https://github.com/open-mmlab/mmaction2/pull/292))
+- Fix the data preparation bug for `something-something` dataset ([#278](https://github.com/open-mmlab/mmaction2/pull/278))
+- Fix the invalid config url in slowonly README data benchmark ([#249](https://github.com/open-mmlab/mmaction2/pull/249))
+- Validate that the performance of models trained with videos have no significant difference comparing to the performance of models trained with rawframes ([#256](https://github.com/open-mmlab/mmaction2/pull/256))
+- Correct the `img_norm_cfg` used by TSN-3seg-R50 UCF-101 model, improve the Top-1 accuracy by 3% ([#273](https://github.com/open-mmlab/mmaction2/pull/273))
+
+**ModelZoo**
+- Add Baselines for Kinetics-600 and Kinetics-700, including TSN-R50-8seg and SlowOnly-R50-8x8 ([#259](https://github.com/open-mmlab/mmaction2/pull/259))
+- Add OmniSource benchmark on MiniKineitcs ([#296](https://github.com/open-mmlab/mmaction2/pull/296))
+- Add Baselines for HVU, including TSN-R18-8seg on 6 tag categories of HVU ([#287](https://github.com/open-mmlab/mmaction2/pull/287))
+- Add X3D models ported from [SlowFast](https://github.com/facebookresearch/SlowFast/) ([#288](https://github.com/open-mmlab/mmaction2/pull/288))
+
+### v0.7.0 (30/9/2020)
+
+**Highlights**
+- Support TPN
+- Support JHMDB, UCF101-24, HVU dataset preparation
+- support onnx model conversion
+
+**New Features**
+- Support the data pre-processing pipeline for the HVU Dataset ([#277](https://github.com/open-mmlab/mmaction2/pull/227/))
+- Support real-time action recognition from web camera ([#171](https://github.com/open-mmlab/mmaction2/pull/171))
+- Support onnx ([#160](https://github.com/open-mmlab/mmaction2/pull/160))
+- Support UCF101-24 preparation ([#219](https://github.com/open-mmlab/mmaction2/pull/219))
+- Support evaluating mAP for ActivityNet with [CUHK17_activitynet_pred](http://activity-net.org/challenges/2017/evaluation.html) ([#176](https://github.com/open-mmlab/mmaction2/pull/176))
+- Add the data pipeline for ActivityNet, including downloading videos, extracting RGB and Flow frames, finetuning TSN and extracting feature ([#190](https://github.com/open-mmlab/mmaction2/pull/190))
+- Support JHMDB preparation ([#220](https://github.com/open-mmlab/mmaction2/pull/220))
+
+**ModelZoo**
+- Add finetuning setting for SlowOnly ([#173](https://github.com/open-mmlab/mmaction2/pull/173))
+- Add TSN and SlowOnly models trained with [OmniSource](https://arxiv.org/abs/2003.13042), which achieve 75.7% Top-1 with TSN-R50-3seg and 80.4% Top-1 with SlowOnly-R101-8x8 ([#215](https://github.com/open-mmlab/mmaction2/pull/215))
+
+**Improvements**
+- Support demo with video url ([#165](https://github.com/open-mmlab/mmaction2/pull/165))
+- Support multi-batch when testing ([#184](https://github.com/open-mmlab/mmaction2/pull/184))
+- Add tutorial for adding a new learning rate updater ([#181](https://github.com/open-mmlab/mmaction2/pull/181))
+- Add config name in meta info ([#183](https://github.com/open-mmlab/mmaction2/pull/183))
+- Remove git hash in `__version__` ([#189](https://github.com/open-mmlab/mmaction2/pull/189))
+- Check mmcv version ([#189](https://github.com/open-mmlab/mmaction2/pull/189))
+- Update url with 'https://download.openmmlab.com' ([#208](https://github.com/open-mmlab/mmaction2/pull/208))
+- Update Docker file to support PyTorch 1.6 and update `install.md` ([#209](https://github.com/open-mmlab/mmaction2/pull/209))
+- Polish readsthedocs display ([#217](https://github.com/open-mmlab/mmaction2/pull/217), [#229](https://github.com/open-mmlab/mmaction2/pull/229))
+
+**Bug Fixes**
+- Fix the bug when using OpenCV to extract only RGB frames with original shape ([#184](https://github.com/open-mmlab/mmaction2/pull/187))
+- Fix the bug of sthv2 `num_classes` from 339 to 174 ([#174](https://github.com/open-mmlab/mmaction2/pull/174), [#207](https://github.com/open-mmlab/mmaction2/pull/207))
+
+
+### v0.6.0 (2/9/2020)
+
+**Highlights**
+- Support TIN, CSN, SSN, NonLocal
+- Support FP16 training
+
+**New Features**
+- Support NonLocal module and provide ckpt in TSM and I3D ([#41](https://github.com/open-mmlab/mmaction2/pull/41))
+- Support SSN ([#33](https://github.com/open-mmlab/mmaction2/pull/33), [#37](https://github.com/open-mmlab/mmaction2/pull/37), [#52](https://github.com/open-mmlab/mmaction2/pull/52), [#55](https://github.com/open-mmlab/mmaction2/pull/55))
+- Support CSN ([#87](https://github.com/open-mmlab/mmaction2/pull/87))
+- Support TIN ([#53](https://github.com/open-mmlab/mmaction2/pull/53))
+- Support HMDB51 dataset preparation ([#60](https://github.com/open-mmlab/mmaction2/pull/60))
+- Support encoding videos from frames ([#84](https://github.com/open-mmlab/mmaction2/pull/84))
+- Support FP16 training ([#25](https://github.com/open-mmlab/mmaction2/pull/25))
+- Enhance demo by supporting rawframe inference ([#59](https://github.com/open-mmlab/mmaction2/pull/59)), output video/gif ([#72](https://github.com/open-mmlab/mmaction2/pull/72))
+
+**ModelZoo**
+- Update Slowfast modelzoo ([#51](https://github.com/open-mmlab/mmaction2/pull/51))
+- Update TSN, TSM video checkpoints ([#50](https://github.com/open-mmlab/mmaction2/pull/50))
+- Add data benchmark for TSN ([#57](https://github.com/open-mmlab/mmaction2/pull/57))
+- Add data benchmark for SlowOnly ([#77](https://github.com/open-mmlab/mmaction2/pull/77))
+- Add BSN/BMN performance results with feature extracted by our codebase ([#99](https://github.com/open-mmlab/mmaction2/pull/99))
+
+**Improvements**
+- Polish data preparation codes ([#70](https://github.com/open-mmlab/mmaction2/pull/70))
+- Improve data preparation scripts ([#58](https://github.com/open-mmlab/mmaction2/pull/58))
+- Improve unittest coverage and minor fix ([#62](https://github.com/open-mmlab/mmaction2/pull/62))
+- Support PyTorch 1.6 in CI ([#117](https://github.com/open-mmlab/mmaction2/pull/117))
+- Support `with_offset` for rawframe dataset ([#48](https://github.com/open-mmlab/mmaction2/pull/48))
+- Support json annotation files ([#119](https://github.com/open-mmlab/mmaction2/pull/119))
+- Support `multi-class` in TSMHead ([#104](https://github.com/open-mmlab/mmaction2/pull/104))
+- Support using `val_step()` to validate data for each `val` workflow ([#123](https://github.com/open-mmlab/mmaction2/pull/123))
+- Use `xxInit()` method to get `total_frames` and make `total_frames` a required key ([#90](https://github.com/open-mmlab/mmaction2/pull/90))
+- Add paper introduction in model readme ([#140](https://github.com/open-mmlab/mmaction2/pull/140))
+- Adjust the directory structure of `tools/` and rename some scripts files ([#142](https://github.com/open-mmlab/mmaction2/pull/142))
+
+**Bug Fixes**
+- Fix configs for localization test ([#67](https://github.com/open-mmlab/mmaction2/pull/67))
+- Fix configs of SlowOnly by fixing lr to 8 gpus ([#136](https://github.com/open-mmlab/mmaction2/pull/136))
+- Fix the bug in analyze_log ([#54](https://github.com/open-mmlab/mmaction2/pull/54))
+- Fix the bug of generating HMDB51 class index file ([#69](https://github.com/open-mmlab/mmaction2/pull/69))
+- Fix the bug of using `load_checkpoint()` in ResNet ([#93](https://github.com/open-mmlab/mmaction2/pull/93))
+- Fix the bug of `--work-dir` when using slurm training script ([#110](https://github.com/open-mmlab/mmaction2/pull/110))
+- Correct the sthv1/sthv2 rawframes filelist generate command ([#71](https://github.com/open-mmlab/mmaction2/pull/71))
+- `CosineAnnealing` typo ([#47](https://github.com/open-mmlab/mmaction2/pull/47))
+
+
+### v0.5.0 (9/7/2020)
+
+**Highlights**
+- MMAction2 is released
+
+**New Features**
+- Support various datasets: UCF101, Kinetics-400, Something-Something V1&V2, Moments in Time,
+ Multi-Moments in Time, THUMOS14
+- Support various action recognition methods: TSN, TSM, R(2+1)D, I3D, SlowOnly, SlowFast, Non-local
+- Support various action localization methods: BSN, BMN
+- Colab demo for action recognition
diff --git a/Downstream/Open-Set-Action-Recognition/docs/conf.py b/Downstream/Open-Set-Action-Recognition/docs/conf.py
new file mode 100644
index 0000000..3aa2ca7
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/conf.py
@@ -0,0 +1,79 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import subprocess
+import sys
+
+sys.path.insert(0, os.path.abspath('..'))
+
+# -- Project information -----------------------------------------------------
+
+project = 'MMAction2'
+copyright = '2020, OpenMMLab'
+author = 'MMAction2 Authors'
+version_file = '../mmaction/version.py'
+
+
+def get_version():
+ with open(version_file, 'r') as f:
+ exec(compile(f.read(), version_file, 'exec'))
+ return locals()['__version__']
+
+
+# The full version, including alpha/beta/rc tags
+release = get_version()
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+ 'sphinx.ext.autodoc', 'sphinx.ext.napoleon', 'sphinx.ext.viewcode',
+ 'recommonmark', 'sphinx_markdown_tables'
+]
+
+# numpy and torch are required
+autodoc_mock_imports = ['mmaction.version', 'cv2', 'PIL']
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+# -- Options for HTML output -------------------------------------------------
+source_suffix = {'.rst': 'restructuredtext', '.md': 'markdown'}
+
+# The theme to use for HTML and HTML Help pages. See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'sphinx_rtd_theme'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = []
+
+master_doc = 'index'
+
+
+def builder_inited_handler(app):
+ subprocess.run(['./merge_docs.sh'])
+ subprocess.run(['./stat.py'])
+
+
+def setup(app):
+ app.connect('builder-inited', builder_inited_handler)
diff --git a/Downstream/Open-Set-Action-Recognition/docs/config.md b/Downstream/Open-Set-Action-Recognition/docs/config.md
new file mode 100644
index 0000000..368d2bc
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/config.md
@@ -0,0 +1,495 @@
+# Config System
+
+We use python files as configs. You can find all the provided configs under `$MMAction2/configs`.
+
+
+
+- [Config File Naming Convention](#config-file-naming-convention)
+- [Config File Structure](#config-file-structure)
+ * [Config System for Action localization](#config-system-for-action-localization)
+ * [Config System for Action Recognition](#config-system-for-action-recognition)
+- [FAQ](#faq)
+ * [Use intermediate variables in configs](#use-intermediate-variables-in-configs)
+
+
+
+## Config File Naming Convention
+
+We follow the style below to name config files. Contributors are advised to follow the same style.
+
+```
+{model}_[model setting]_{backbone}_[misc]_{data setting}_[gpu x batch_per_gpu]_{schedule}_{dataset}_{modality}
+```
+
+`{xxx}` is required field and `[yyy]` is optional.
+
+- `{model}`: model type, e.g. `tsn`, `i3d`, etc.
+- `[model setting]`: specific setting for some models.
+- `{backbone}`: backbone type, e.g. `r50` (ResNet-50), etc.
+- `[misc]`: miscellaneous setting/plugins of model, e.g. `dense`, `320p`, `video`, etc.
+- `{data setting}`: frame sample setting in `{clip_len}x{frame_interval}x{num_clips}` format.
+- `[gpu x batch_per_gpu]`: GPUs and samples per GPU.
+- `{schedule}`: training schedule, e.g. `20e` means 20 epochs.
+- `{dataset}`: dataset name, e.g. `kinetics400`, `mmit`, etc.
+- `{modality}`: frame modality, e.g. `rgb`, `flow`, etc.
+
+## Config File Structure
+
+Please refer to the corresponding pages for config file structure for different tasks.
+
+### Config System for Action localization
+
+We incorporate modular design into our config system,
+which is convenient to conduct various experiments.
+
+- An Example of BMN
+
+ To help the users have a basic idea of a complete config structure and the modules in an action localization system,
+ we make brief comments on the config of BMN as the following.
+ For more detailed usage and alternative for per parameter in each module, please refer to the API documentation.
+
+ ```python
+ # model settings
+ model = dict( # Config of the model
+ type='BMN', # Type of the localizer
+ temporal_dim=100, # Total frames selected for each video
+ boundary_ratio=0.5, # Ratio for determining video boundaries
+ num_samples=32, # Number of samples for each proposal
+ num_samples_per_bin=3, # Number of bin samples for each sample
+ feat_dim=400, # Dimension of feature
+ soft_nms_alpha=0.4, # Soft NMS alpha
+ soft_nms_low_threshold=0.5, # Soft NMS low threshold
+ soft_nms_high_threshold=0.9, # Soft NMS high threshold
+ post_process_top_k=100) # Top k proposals in post process
+ # model training and testing settings
+ train_cfg = None # Config of training hyperparameters for BMN
+ test_cfg = dict(average_clips='score') # Config for testing hyperparameters for BMN
+
+ # dataset settings
+ dataset_type = 'ActivityNetDataset' # Type of dataset for training, valiation and testing
+ data_root = 'data/activitynet_feature_cuhk/csv_mean_100/' # Root path to data for training
+ data_root_val = 'data/activitynet_feature_cuhk/csv_mean_100/' # Root path to data for validation and testing
+ ann_file_train = 'data/ActivityNet/anet_anno_train.json' # Path to the annotation file for training
+ ann_file_val = 'data/ActivityNet/anet_anno_val.json' # Path to the annotation file for validation
+ ann_file_test = 'data/ActivityNet/anet_anno_test.json' # Path to the annotation file for testing
+
+ train_pipeline = [ # List of training pipeline steps
+ dict(type='LoadLocalizationFeature'), # Load localization feature pipeline
+ dict(type='GenerateLocalizationLabels'), # Generate localization labels pipeline
+ dict( # Config of Collect
+ type='Collect', # Collect pipeline that decides which keys in the data should be passed to the localizer
+ keys=['raw_feature', 'gt_bbox'], # Keys of input
+ meta_name='video_meta', # Meta name
+ meta_keys=['video_name']), # Meta keys of input
+ dict( # Config of ToTensor
+ type='ToTensor', # Convert other types to tensor type pipeline
+ keys=['raw_feature']), # Keys to be converted from image to tensor
+ dict( # Config of ToDataContainer
+ type='ToDataContainer', # Pipeline to convert the data to DataContainer
+ fields=[dict(key='gt_bbox', stack=False, cpu_only=True)]) # Required fields to be converted with keys and attributes
+ ]
+ val_pipeline = [ # List of validation pipeline steps
+ dict(type='LoadLocalizationFeature'), # Load localization feature pipeline
+ dict(type='GenerateLocalizationLabels'), # Generate localization labels pipeline
+ dict( # Config of Collect
+ type='Collect', # Collect pipeline that decides which keys in the data should be passed to the localizer
+ keys=['raw_feature', 'gt_bbox'], # Keys of input
+ meta_name='video_meta', # Meta name
+ meta_keys=[
+ 'video_name', 'duration_second', 'duration_frame', 'annotations',
+ 'feature_frame'
+ ]), # Meta keys of input
+ dict( # Config of ToTensor
+ type='ToTensor', # Convert other types to tensor type pipeline
+ keys=['raw_feature']), # Keys to be converted from image to tensor
+ dict( # Config of ToDataContainer
+ type='ToDataContainer', # Pipeline to convert the data to DataContainer
+ fields=[dict(key='gt_bbox', stack=False, cpu_only=True)]) # Required fields to be converted with keys and attributes
+ ]
+ test_pipeline = [ # List of testing pipeline steps
+ dict(type='LoadLocalizationFeature'), # Load localization feature pipeline
+ dict( # Config of Collect
+ type='Collect', # Collect pipeline that decides which keys in the data should be passed to the localizer
+ keys=['raw_feature'], # Keys of input
+ meta_name='video_meta', # Meta name
+ meta_keys=[
+ 'video_name', 'duration_second', 'duration_frame', 'annotations',
+ 'feature_frame'
+ ]), # Meta keys of input
+ dict( # Config of ToTensor
+ type='ToTensor', # Convert other types to tensor type pipeline
+ keys=['raw_feature']), # Keys to be converted from image to tensor
+ ]
+ data = dict( # Config of data
+ videos_per_gpu=8, # Batch size of each single GPU
+ workers_per_gpu=8, # Workers to pre-fetch data for each single GPU
+ train_dataloader=dict( # Additional config of train dataloader
+ drop_last=True), # Whether to drop out the last batch of data in training
+ val_dataloader=dict( # Additional config of validation dataloader
+ videos_per_gpu=1), # Batch size of each single GPU during evaluation
+ test_dataloader=dict( # Additional config of test dataloader
+ videos_per_gpu=2), # Batch size of each single GPU during testing
+ test=dict( # Testing dataset config
+ type=dataset_type,
+ ann_file=ann_file_test,
+ pipeline=test_pipeline,
+ data_prefix=data_root_val),
+ val=dict( # Validation dataset config
+ type=dataset_type,
+ ann_file=ann_file_val,
+ pipeline=val_pipeline,
+ data_prefix=data_root_val),
+ train=dict( # Training dataset config
+ type=dataset_type,
+ ann_file=ann_file_train,
+ pipeline=train_pipeline,
+ data_prefix=data_root))
+
+ # optimizer
+ optimizer = dict(
+ # Config used to build optimizer, support (1). All the optimizers in PyTorch
+ # whose arguments are also the same as those in PyTorch. (2). Custom optimizers
+ # which are builed on `constructor`, referring to "tutorials/new_modules.md"
+ # for implementation.
+ type='Adam', # Type of optimizer, refer to https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/optimizer/default_constructor.py#L13 for more details
+ lr=0.001, # Learning rate, see detail usages of the parameters in the documentaion of PyTorch
+ weight_decay=0.0001) # Weight decay of Adam
+ optimizer_config = dict( # Config used to build the optimizer hook
+ grad_clip=None) # Most of the methods do not use gradient clip
+ # learning policy
+ lr_config = dict( # Learning rate scheduler config used to register LrUpdater hook
+ policy='step', # Policy of scheduler, also support CosineAnnealing, Cyclic, etc. Refer to details of supported LrUpdater from https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/lr_updater.py#L9
+ step=7) # Steps to decay the learning rate
+
+ total_epochs = 9 # Total epochs to train the model
+ checkpoint_config = dict( # Config to set the checkpoint hook, Refer to https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/checkpoint.py for implementation
+ interval=1) # Interval to save checkpoint
+ evaluation = dict( # Config of evaluation during training
+ interval=1, # Interval to perform evaluation
+ metrics=['AR@AN']) # Metrics to be performed
+ log_config = dict( # Config to register logger hook
+ interval=50, # Interval to print the log
+ hooks=[ # Hooks to be implemented during training
+ dict(type='TextLoggerHook'), # The logger used to record the training process
+ # dict(type='TensorboardLoggerHook'), # The Tensorboard logger is also supported
+ ])
+
+ # runtime settings
+ dist_params = dict(backend='nccl') # Parameters to setup distributed training, the port can also be set
+ log_level = 'INFO' # The level of logging
+ work_dir = './work_dirs/bmn_400x100_2x8_9e_activitynet_feature/' # Directory to save the model checkpoints and logs for the current experiments
+ load_from = None # load models as a pre-trained model from a given path. This will not resume training
+ resume_from = None # Resume checkpoints from a given path, the training will be resumed from the epoch when the checkpoint's is saved
+ workflow = [('train', 1)] # Workflow for # runner. [('train', 1)] means there is only one workflow and the workflow named 'train' is executed once
+ output_config = dict( # Config of localization ouput
+ out=f'{work_dir}/results.json', # Path to output file
+ output_format='json') # File format of output file
+ ```
+
+### Config System for Action Recognition
+
+We incorporate modular design into our config system,
+which is convenient to conduct various experiments.
+
+- An Example of TSN
+
+ To help the users have a basic idea of a complete config structure and the modules in an action recognition system,
+ we make brief comments on the config of TSN as the following.
+ For more detailed usage and alternative for per parameter in each module, please refer to the API documentation.
+
+ ```python
+ # model settings
+ model = dict( # Config of the model
+ type='Recognizer2D', # Type of the recognizer
+ backbone=dict( # Dict for backbone
+ type='ResNet', # Name of the backbone
+ pretrained='torchvision://resnet50', # The url/site of the pretrained model
+ depth=50, # Depth of ResNet model
+ norm_eval=False), # Whether to set BN layers to eval mode when training
+ cls_head=dict( # Dict for classification head
+ type='TSNHead', # Name of classification head
+ num_classes=400, # Number of classes to be classified.
+ in_channels=2048, # The input channels of classification head.
+ spatial_type='avg', # Type of pooling in spatial dimension
+ consensus=dict(type='AvgConsensus', dim=1), # Config of consensus module
+ dropout_ratio=0.4, # Probability in dropout layer
+ init_std=0.01)) # Std value for linear layer initiation
+ # model training and testing settings
+ train_cfg = None # Config of training hyperparameters for TSN
+ test_cfg = dict(average_clips=None) # Config for testing hyperparameters for TSN. Here we define clip averaging method in it
+
+ # dataset settings
+ dataset_type = 'RawframeDataset' # Type of dataset for training, valiation and testing
+ data_root = 'data/kinetics400/rawframes_train/' # Root path to data for training
+ data_root_val = 'data/kinetics400/rawframes_val/' # Root path to data for validation and testing
+ ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt' # Path to the annotation file for training
+ ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt' # Path to the annotation file for validation
+ ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt' # Path to the annotation file for testing
+ img_norm_cfg = dict( # Config of image normalition used in data pipeline
+ mean=[123.675, 116.28, 103.53], # Mean values of different channels to normalize
+ std=[58.395, 57.12, 57.375], # Std values of different channels to normalize
+ to_bgr=False) # Whether to convert channels from RGB to BGR
+
+ train_pipeline = [ # List of training pipeline steps
+ dict( # Config of SampleFrames
+ type='SampleFrames', # Sample frames pipeline, sampling frames from video
+ clip_len=1, # Frames of each sampled output clip
+ frame_interval=1, # Temporal interval of adjacent sampled frames
+ num_clips=3), # Number of clips to be sampled
+ dict( # Config of RawFrameDecode
+ type='RawFrameDecode'), # Load and decode Frames pipeline, picking raw frames with given indices
+ dict( # Config of Resize
+ type='Resize', # Resize pipeline
+ scale=(-1, 256)), # The scale to resize images
+ dict( # Config of MultiScaleCrop
+ type='MultiScaleCrop', # Multi scale crop pipeline, cropping images with a list of randomly selected scales
+ input_size=224, # Input size of the network
+ scales=(1, 0.875, 0.75, 0.66), # Scales of weight and height to be selected
+ random_crop=False, # Whether to randomly sample cropping bbox
+ max_wh_scale_gap=1), # Maximum gap of w and h scale levels
+ dict( # Config of Resize
+ type='Resize', # Resize pipeline
+ scale=(224, 224), # The scale to resize images
+ keep_ratio=False), # Whether to resize with changing the aspect ratio
+ dict( # Config of Flip
+ type='Flip', # Flip Pipeline
+ flip_ratio=0.5), # Probability of implementing flip
+ dict( # Config of Normalize
+ type='Normalize', # Normalize pipeline
+ **img_norm_cfg), # Config of image normalization
+ dict( # Config of FormatShape
+ type='FormatShape', # Format shape pipeline, Format final image shape to the given input_format
+ input_format='NCHW'), # Final image shape format
+ dict( # Config of Collect
+ type='Collect', # Collect pipeline that decides which keys in the data should be passed to the recognizer
+ keys=['imgs', 'label'], # Keys of input
+ meta_keys=[]), # Meta keys of input
+ dict( # Config of ToTensor
+ type='ToTensor', # Convert other types to tensor type pipeline
+ keys=['imgs', 'label']) # Keys to be converted from image to tensor
+ ]
+ val_pipeline = [ # List of validation pipeline steps
+ dict( # Config of SampleFrames
+ type='SampleFrames', # Sample frames pipeline, sampling frames from video
+ clip_len=1, # Frames of each sampled output clip
+ frame_interval=1, # Temporal interval of adjacent sampled frames
+ num_clips=3, # Number of clips to be sampled
+ test_mode=True), # Whether to set test mode in sampling
+ dict( # Config of RawFrameDecode
+ type='RawFrameDecode'), # Load and decode Frames pipeline, picking raw frames with given indices
+ dict( # Config of Resize
+ type='Resize', # Resize pipeline
+ scale=(-1, 256)), # The scale to resize images
+ dict( # Config of CenterCrop
+ type='CenterCrop', # Center crop pipeline, cropping the center area from images
+ crop_size=224), # The size to crop images
+ dict( # Config of Flip
+ type='Flip', # Flip pipeline
+ flip_ratio=0), # Probability of implementing flip
+ dict( # Config of Normalize
+ type='Normalize', # Normalize pipeline
+ **img_norm_cfg), # Config of image normalization
+ dict( # Config of FormatShape
+ type='FormatShape', # Format shape pipeline, Format final image shape to the given input_format
+ input_format='NCHW'), # Final image shape format
+ dict( # Config of Collect
+ type='Collect', # Collect pipeline that decides which keys in the data should be passed to the recognizer
+ keys=['imgs', 'label'], # Keys of input
+ meta_keys=[]), # Meta keys of input
+ dict( # Config of ToTensor
+ type='ToTensor', # Convert other types to tensor type pipeline
+ keys=['imgs']) # Keys to be converted from image to tensor
+ ]
+ test_pipeline = [ # List of testing pipeline steps
+ dict( # Config of SampleFrames
+ type='SampleFrames', # Sample frames pipeline, sampling frames from video
+ clip_len=1, # Frames of each sampled output clip
+ frame_interval=1, # Temporal interval of adjacent sampled frames
+ num_clips=25, # Number of clips to be sampled
+ test_mode=True), # Whether to set test mode in sampling
+ dict( # Config of RawFrameDecode
+ type='RawFrameDecode'), # Load and decode Frames pipeline, picking raw frames with given indices
+ dict( # Config of Resize
+ type='Resize', # Resize pipeline
+ scale=(-1, 256)), # The scale to resize images
+ dict( # Config of CenterCrop
+ type='TenCrop', # Center crop pipeline, cropping the center area from images
+ crop_size=224), # The size to crop images
+ dict( # Config of Flip
+ type='Flip', # Flip pipeline
+ flip_ratio=0), # Probability of implementing flip
+ dict( # Config of Normalize
+ type='Normalize', # Normalize pipeline
+ **img_norm_cfg), # Config of image normalization
+ dict( # Config of FormatShape
+ type='FormatShape', # Format shape pipeline, Format final image shape to the given input_format
+ input_format='NCHW'), # Final image shape format
+ dict( # Config of Collect
+ type='Collect', # Collect pipeline that decides which keys in the data should be passed to the recognizer
+ keys=['imgs', 'label'], # Keys of input
+ meta_keys=[]), # Meta keys of input
+ dict( # Config of ToTensor
+ type='ToTensor', # Convert other types to tensor type pipeline
+ keys=['imgs']) # Keys to be converted from image to tensor
+ ]
+ data = dict( # Config of data
+ videos_per_gpu=32, # Batch size of each single GPU
+ workers_per_gpu=4, # Workers to pre-fetch data for each single GPU
+ train_dataloader=dict( # Additional config of train dataloader
+ drop_last=True), # Whether to drop out the last batch of data in training
+ val_dataloader=dict( # Additional config of validation dataloader
+ videos_per_gpu=1), # Batch size of each single GPU during evaluation
+ test_dataloader=dict( # Additional config of test dataloader
+ videos_per_gpu=2), # Batch size of each single GPU during testing
+ train=dict( # Training dataset config
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict( # Validation dataset config
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict( # Testing dataset config
+ type=dataset_type,
+ ann_file=ann_file_test,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+ # optimizer
+ optimizer = dict(
+ # Config used to build optimizer, support (1). All the optimizers in PyTorch
+ # whose arguments are also the same as those in PyTorch. (2). Custom optimizers
+ # which are builed on `constructor`, referring to "tutorials/new_modules.md"
+ # for implementation.
+ type='SGD', # Type of optimizer, refer to https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/optimizer/default_constructor.py#L13 for more details
+ lr=0.01, # Learning rate, see detail usages of the parameters in the documentaion of PyTorch
+ momentum=0.9, # Momentum,
+ weight_decay=0.0001) # Weight decay of SGD
+ optimizer_config = dict( # Config used to build the optimizer hook
+ grad_clip=dict(max_norm=40, norm_type=2)) # Use gradient clip
+ # learning policy
+ lr_config = dict( # Learning rate scheduler config used to register LrUpdater hook
+ policy='step', # Policy of scheduler, also support CosineAnnealing, Cyclic, etc. Refer to details of supported LrUpdater from https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/lr_updater.py#L9
+ step=[40, 80]) # Steps to decay the learning rate
+ total_epochs = 100 # Total epochs to train the model
+ checkpoint_config = dict( # Config to set the checkpoint hook, Refer to https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/checkpoint.py for implementation
+ interval=5) # Interval to save checkpoint
+ evaluation = dict( # Config of evaluation during training
+ interval=5, # Interval to perform evaluation
+ metrics=['top_k_accuracy', 'mean_class_accuracy'], # Metrics to be performed
+ topk=(1, 5)) # K value for `top_k_accuracy` metric
+ log_config = dict( # Config to register logger hook
+ interval=20, # Interval to print the log
+ hooks=[ # Hooks to be implemented during training
+ dict(type='TextLoggerHook'), # The logger used to record the training process
+ # dict(type='TensorboardLoggerHook'), # The Tensorboard logger is also supported
+ ])
+
+ # runtime settings
+ dist_params = dict(backend='nccl') # Parameters to setup distributed training, the port can also be set
+ log_level = 'INFO' # The level of logging
+ work_dir = './work_dirs/tsn_r50_1x1x3_100e_kinetics400_rgb/' # Directory to save the model checkpoints and logs for the current experiments
+ load_from = None # load models as a pre-trained model from a given path. This will not resume training
+ resume_from = None # Resume checkpoints from a given path, the training will be resumed from the epoch when the checkpoint's is saved
+ workflow = [('train', 1)] # Workflow for runner. [('train', 1)] means there is only one workflow and the workflow named 'train' is executed once
+
+ ```
+
+
+## FAQ
+
+### Use intermediate variables in configs
+
+Some intermediate variables are used in the config files, like `train_pipeline`/`val_pipeline`/`test_pipeline`,
+`ann_file_train`/`ann_file_val`/`ann_file_test`, `img_norm_cfg` etc.
+
+For Example, we would like to first define `train_pipeline`/`val_pipeline`/`test_pipeline` and pass them into `data`.
+Thus, `train_pipeline`/`val_pipeline`/`test_pipeline` are intermediate variable.
+
+we also define `ann_file_train`/`ann_file_val`/`ann_file_test` and `data_root`/`data_root_val` to provide data pipeline some
+basic information.
+
+In addition, we use `img_norm_cfg` as intermediate variables to construct data augmentation components.
+
+```python
+...
+dataset_type = 'RawframeDataset'
+data_root = 'data/kinetics400/rawframes_train'
+data_root_val = 'data/kinetics400/rawframes_val'
+ann_file_train = 'data/kinetics400/kinetics400_train_list_rawframes.txt'
+ann_file_val = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+ann_file_test = 'data/kinetics400/kinetics400_val_list_rawframes.txt'
+
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.8),
+ random_crop=False,
+ max_wh_scale_gap=0),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=1,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=32,
+ frame_interval=2,
+ num_clips=10,
+ test_mode=True),
+ dict(type='RawFrameDecode'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='ThreeCrop', crop_size=256),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+
+data = dict(
+ videos_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ data_prefix=data_root,
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=val_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ data_prefix=data_root_val,
+ pipeline=test_pipeline))
+```
diff --git a/Downstream/Open-Set-Action-Recognition/docs/data_preparation.md b/Downstream/Open-Set-Action-Recognition/docs/data_preparation.md
new file mode 100644
index 0000000..9197031
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/data_preparation.md
@@ -0,0 +1,153 @@
+# Data Preparation
+
+We provide some tips for MMAction2 data preparation in this file.
+
+
+
+- [Notes on Video Data Format](#notes-on-video-data-format)
+- [Getting Data](#getting-data)
+ * [Prepare videos](#prepare-videos)
+ * [Extract frames](#extract-frames)
+ + [Alternative to denseflow](#alternative-to-denseflow)
+ * [Generate file list](#generate-file-list)
+ * [Prepare audio](#prepare-audio)
+
+
+
+## Notes on Video Data Format
+
+MMAction2 supports two types of data format: raw frames and video. The former is widely used in previous projects such as [TSN](https://github.com/yjxiong/temporal-segment-networks).
+This is fast when SSD is available but fails to scale to the fast-growing datasets.
+(For example, the newest edition of [Kinetics](https://deepmind.com/research/open-source/open-source-datasets/kinetics/) has 650K videos and the total frames will take up several TBs.)
+The latter saves much space but has to do the computation intensive video decoding at execution time
+To make video decoding faster, we support several efficient video loading libraries, such as [decord](https://github.com/zhreshold/decord), [PyAV](https://github.com/PyAV-Org/PyAV), etc.
+
+## Getting Data
+
+The following guide is helpful when you want to experiment with custom dataset.
+Similar to the datasets stated above, it is recommended organizing in `$MMACTION2/data/$DATASET`.
+
+### Prepare videos
+
+ Please refer to the official website and/or the official script to prepare the videos.
+Note that the videos should be arranged in either
+
+(1). A two-level directory organized by `${CLASS_NAME}/${VIDEO_ID}`, which is recommended to be used for for action recognition datasets (such as UCF101 and Kinetics)
+
+(2). A single-level directory, which is recommended to be used for for action detection datasets or those with multiple annotations per video (such as THUMOS14).
+
+### Extract frames
+
+To extract both frames and optical flow, you can use the tool [denseflow](https://github.com/open-mmlab/denseflow) we wrote.
+Since different frame extraction tools produce different number of frames,
+it is beneficial to use the same tool to do both frame extraction and the flow computation, to avoid mismatching of frame counts.
+
+```shell
+python build_rawframes.py ${SRC_FOLDER} ${OUT_FOLDER} [--task ${TASK}] [--level ${LEVEL}] \
+ [--num-worker ${NUM_WORKER}] [--flow-type ${FLOW_TYPE}] [--out-format ${OUT_FORMAT}] \
+ [--ext ${EXT}] [--new-width ${NEW_WIDTH}] [--new-height ${NEW_HEIGHT}] [--new-short ${NEW_SHORT}] \
+ [--resume] [--use-opencv] [--mixed-ext]
+```
+
+- `SRC_FOLDER`: Folder of the original video.
+- `OUT_FOLDER`: Root folder where the extracted frames and optical flow store.
+- `TASK`: Extraction task indicating which kind of frames to extract. Allowed choices are `rgb`, `flow`, `both`.
+- `LEVEL`: Directory level. 1 for the single-level directory or 2 for the two-level directory.
+- `NUM_WORKER`: Number of workers to build rawframes.
+- `FLOW_TYPE`: Flow type to extract, e.g., `None`, `tvl1`, `warp_tvl1`, `farn`, `brox`.
+- `OUT_FORMAT`: Output format for extracted frames, e.g., `jpg`, `h5`, `png`.
+- `EXT`: Video file extension, e.g., `avi`, `mp4`.
+- `NEW_WIDTH`: Resized image width of output.
+- `NEW_HEIGHT`: Resized image height of output.
+- `NEW_SHORT`: Resized image short side length keeping ratio.
+- `--resume`: Whether to resume optical flow extraction instead of overwriting.
+- `--use-opencv`: Whether to use OpenCV to extract rgb frames.
+- `--mixed-ext`: Indicate whether process video files with mixed extensions.
+
+The recommended practice is
+
+1. set `$OUT_FOLDER` to be a folder located in SSD.
+2. symlink the link `$OUT_FOLDER` to `$MMACTION2/data/$DATASET/rawframes`.
+
+```shell
+ln -s ${YOUR_FOLDER} $MMACTION2/data/$DATASET/rawframes
+```
+
+#### Alternative to denseflow
+
+In case your device doesn't fulfill the installation requirement of [denseflow](https://github.com/open-mmlab/denseflow)(like Nvidia driver version), or you just want to see some quick demos about flow extraction, we provide a python script `tools/flow_extraction.py` as an alternative to denseflow. You can use it for rgb frames and optical flow extraction from one or several videos. Note that the speed of the script is much slower than denseflow, since it runs optical flow algorithms on CPU.
+
+```shell
+python tools/flow_extraction.py --input ${INPUT} [--prefix ${PREFIX}] [--dest ${DEST}] [--rgb-tmpl ${RGB_TMPL}] \
+ [--flow-tmpl ${FLOW_TMPL}] [--start-idx ${START_IDX}] [--method ${METHOD}] [--bound ${BOUND}] [--save-rgb]
+```
+
+- `INPUT`: Videos for frame extraction, can be single video or a video list, the video list should be a txt file and just consists of filenames without directories.
+- `PREFIX`: The prefix of input videos, used when input is a video list.
+- `DEST`: The destination to save extracted frames.
+- `RGB_TMPL`: The template filename of rgb frames.
+- `FLOW_TMPL`: The template filename of flow frames.
+- `START_IDX`: The start index of extracted frames.
+- `METHOD`: The method used to generate flow.
+- `BOUND`: The maximum of optical flow.
+- `SAVE_RGB`: Also save extracted rgb frames.
+
+### Generate file list
+
+We provide a convenient script to generate annotation file list. You can use the following command to extract frames.
+
+```shell
+cd $MMACTION2
+python tools/data/build_file_list.py ${DATASET} ${SRC_FOLDER} [--rgb-prefix ${RGB_PREFIX}] \
+ [--flow-x-prefix ${FLOW_X_PREFIX}] [--flow-y-prefix ${FLOW_Y_PREFIX}] [--num-split ${NUM_SPLIT}] \
+ [--subset ${SUBSET}] [--level ${LEVEL}] [--format ${FORMAT}] [--out-root-path ${OUT_ROOT_PATH}] \
+ [--seed ${SEED}] [--shuffle]
+```
+
+- `DATASET`: Dataset to be prepared, e.g., `ucf101`, `kinetics400`, `thumos14`, `sthv1`, `sthv2`, etc.
+- `SRC_FOLDER`: Folder of the corresponding data format:
+ - "$MMACTION2/data/$DATASET/rawframes" if `--format rawframes`.
+ - "$MMACTION2/data/$DATASET/videos" if `--format videos`.
+- `RGB_PREFIX`: Name prefix of rgb frames.
+- `FLOW_X_PREFIX`: Name prefix of x flow frames.
+- `FLOW_Y_PREFIX`: Name prefix of y flow frames.
+- `NUM_SPLIT`: Number of split to file list.
+- `SUBSET`: Subset to generate file list. Allowed choice are `train`, `val`, `test`.
+- `LEVEL`: Directory level. 1 for the single-level directory or 2 for the two-level directory.
+- `FORMAT`: Source data format to generate file list. Allowed choices are `rawframes`, `videos`.
+- `OUT_ROOT_PATH`: Root path for output
+- `SEED`: Random seed.
+- `--shuffle`: Whether to shuffle the file list.
+
+Now, you can go to [getting_started.md](getting_started.md) to train and test the model.
+
+### Prepare audio
+
+We also provide a simple script for audio waveform extraction and mel-spectrogram generation.
+
+```shell
+cd $MMACTION2
+python tools/data/extract_audio.py ${ROOT} ${DST_ROOT} [--ext ${EXT}] [--num-workers ${N_WORKERS}] \
+ [--level ${LEVEL}]
+```
+
+- `ROOT`: The root directory of the videos.
+- `DST_ROOT`: The destination root directory of the audios.
+- `EXT`: Extention of the video files. e.g., `.mp4`.
+- `N_WORKERS`: Number of processes to be used.
+
+After extracting audios, you are free to decode and generate the spectrogram on-the-fly such as [this](/configs/audio_recognition/tsn_r50_64x1x1_kinetics400_audio.py). As for the annotations, you can directly use those of the rawframes as long as you keep the relative position of audio files same as the rawframes directory. However, extracting spectrogram on-the-fly is slow and bad for prototype iteration. Therefore, we also provide a script (and many useful tools to play with) for you to generation spectrogram off-line.
+
+```shell
+cd $MMACTION2
+python tools/data/build_audio_features.py ${AUDIO_HOME_PATH} ${SPECTROGRAM_SAVE_PATH} [--level ${LEVEL}] \
+ [--ext $EXT] [--num-workers $N_WORKERS] [--part $PART]
+```
+
+- `AUDIO_HOME_PATH`: The root directory of the audio files.
+- `SPECTROGRAM_SAVE_PATH`: The destination root directory of the audio features.
+- `EXT`: Extention of the audio files. e.g., `.m4a`.
+- `N_WORKERS`: Number of processes to be used.
+- `PART`: Determines how many parts to be splited and which part to run. e.g., `2/5` means splitting all files into 5-fold and executing the 2nd part. This is useful if you have several machines.
+
+The annotations for audio spectrogram features are identical to those of rawframes. You can simply make a copy of `dataset_[train/val]_list_rawframes.txt` and rename it as `dataset_[train/val]_list_audio_feature.txt`
diff --git a/Downstream/Open-Set-Action-Recognition/docs/faq.md b/Downstream/Open-Set-Action-Recognition/docs/faq.md
new file mode 100644
index 0000000..3047975
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/faq.md
@@ -0,0 +1,55 @@
+# FAQ
+
+We list some common issues faced by many users and their corresponding solutions here.
+Feel free to enrich the list if you find any frequent issues and have ways to help others to solve them.
+If the contents here do not cover your issue, please create an issue using the [provided templates](/.github/ISSUE_TEMPLATE/error-report.md) and make sure you fill in all required information in the template.
+
+
+## Installation
+
+- **"No module named 'mmcv.ops'"; "No module named 'mmcv._ext'"**
+
+ 1. Uninstall existing mmcv in the environment using `pip uninstall mmcv`.
+ 2. Install mmcv-full following the [installation instruction](https://mmcv.readthedocs.io/en/latest/#installation).
+
+- **"OSError: MoviePy Error: creation of None failed because of the following error"**
+
+ Refer to [install.md](https://github.com/open-mmlab/mmaction2/blob/master/docs/install.md#requirements)
+ 1. For Windows users, [ImageMagick](https://www.imagemagick.org/script/index.php) will not be automatically detected by MoviePy,
+ there is a need to modify `moviepy/config_defaults.py` file by providing the path to the ImageMagick binary called `magick`,
+ like `IMAGEMAGICK_BINARY = "C:\\Program Files\\ImageMagick_VERSION\\magick.exe"`
+ 2. For Linux users, there is a need to modify the `/etc/ImageMagick-6/policy.xml` file by commenting out
+ ` ` to ``,
+ if ImageMagick is not detected by moviepy.
+
+## Data
+
+- **FileNotFound like `No such file or directory: xxx/xxx/img_00300.jpg`**
+
+ In our repo, we set `start_index=1` as default value for rawframe dataset, and `start_index=0` as default value for video dataset.
+ If users encounter FileNotFound error for the first or last frame of the data, there is a need to check the files begin with offset 0 or 1,
+ that is `xxx_00000.jpg` or `xxx_00001.jpg`, and then change the `start_index` value of data pipeline in configs.
+
+
+## Training
+
+- **How to just use trained recognizer models for backbone pre-training ?**
+
+ Refer to [Use Pre-Trained Model](https://github.com/open-mmlab/mmaction2/blob/master/docs/tutorials/finetune.md#use-pre-trained-model),
+ in order to use the pre-trained model for the whole network, the new config adds the link of pre-trained models in the `load_from`.
+
+ And to use backbone for pre-training, you can change `pretrained` value in the backbone dict of config files to the checkpoint path / url.
+ When training, the unexpected keys will be ignored.
+
+
+## Testing
+
+- **How to make predicted score normalized by softmax within [0, 1] ?**
+
+ change this in the config, make `test_cfg = dict(average_clips='prob')`.
+
+## Deploying
+
+- **Why is the onnx model converted by mmaction2 throwing error when converting to other frameworks such as TensorRT?**
+
+ For now, we can only make sure that models in mmaction2 are onnx-compatible. However, some operations in onnx may be unsupported by your target framework for deployment, e.g. TensorRT in [this issue](https://github.com/open-mmlab/mmaction2/issues/414). When such situation occurs, we suggest you raise an issue in the repo of your target framework as long as `pytorch2onnx.py` works well and is verified numerically.
diff --git a/Downstream/Open-Set-Action-Recognition/docs/getting_started.md b/Downstream/Open-Set-Action-Recognition/docs/getting_started.md
new file mode 100644
index 0000000..0bc61b6
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/getting_started.md
@@ -0,0 +1,420 @@
+# Getting Started
+
+This page provides basic tutorials about the usage of MMAction2.
+For installation instructions, please see [install.md](install.md).
+
+
+
+- [Datasets](#datasets)
+- [Inference with Pre-Trained Models](#inference-with-pre-trained-models)
+ * [Test a dataset](#test-a-dataset)
+ * [High-level APIs for testing a video and rawframes.](#high-level-apis-for-testing-a-video-and-rawframes)
+- [Build a Model](#build-a-model)
+ * [Build a model with basic components](#build-a-model-with-basic-components)
+ * [Write a new model](#write-a-new-model)
+- [Train a Model](#train-a-model)
+ * [Iteration pipeline](#iteration-pipeline)
+ * [Training setting](#training-setting)
+ * [Train with a single GPU](#train-with-a-single-gpu)
+ * [Train with multiple GPUs](#train-with-multiple-gpus)
+ * [Train with multiple machines](#train-with-multiple-machines)
+ * [Launch multiple jobs on a single machine](#launch-multiple-jobs-on-a-single-machine)
+- [Tutorials](#tutorials)
+
+
+
+## Datasets
+
+It is recommended to symlink the dataset root to `$MMACTION2/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
+
+```
+mmaction2
+├── mmaction
+├── tools
+├── configs
+├── data
+│ ├── kinetics400
+│ │ ├── rawframes_train
+│ │ ├── rawframes_val
+│ │ ├── kinetics_train_list.txt
+│ │ ├── kinetics_val_list.txt
+│ ├── ucf101
+│ │ ├── rawframes_train
+│ │ ├── rawframes_val
+│ │ ├── ucf101_train_list.txt
+│ │ ├── ucf101_val_list.txt
+│ ├── ...
+```
+For more information on data preparation, please see [data_preparation.md](data_preparation.md)
+
+For using custom datasets, please refer to [Tutorial 2: Adding New Dataset](tutorials/new_dataset.md)
+
+## Inference with Pre-Trained Models
+
+We provide testing scripts to evaluate a whole dataset (Kinetics-400, Something-Something V1&V2, (Multi-)Moments in Time, etc.),
+and provide some high-level apis for easier integration to other projects.
+
+### Test a dataset
+
+- [x] single GPU
+- [x] single node multiple GPUs
+- [x] multiple node
+
+You can use the following commands to test a dataset.
+
+```shell
+# single-gpu testing
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [--out ${RESULT_FILE}] [--eval ${EVAL_METRICS}] \
+ [--gpu-collect] [--tmpdir ${TMPDIR}] [--options ${OPTIONS}] [--average-clips ${AVG_TYPE}] \
+ [--launcher ${JOB_LAUNCHER}] [--local_rank ${LOCAL_RANK}]
+
+# multi-gpu testing
+./tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${GPU_NUM} [--out ${RESULT_FILE}] [--eval ${EVAL_METRICS}] \
+ [--gpu-collect] [--tmpdir ${TMPDIR}] [--options ${OPTIONS}] [--average-clips ${AVG_TYPE}] \
+ [--launcher ${JOB_LAUNCHER}] [--local_rank ${LOCAL_RANK}]
+```
+
+Optional arguments:
+- `RESULT_FILE`: Filename of the output results. If not specified, the results will not be saved to a file.
+- `EVAL_METRICS`: Items to be evaluated on the results. Allowed values depend on the dataset, e.g., `top_k_accuracy`, `mean_class_accuracy` are available for all datasets in recognition, `mmit_mean_average_precision` for Multi-Moments in Time, `mean_average_precision` for Multi-Moments in Time and HVU single category. `AR@AN` for ActivityNet, etc.
+- `--gpu-collect`: If specified, recognition results will be collected using gpu communication. Otherwise, it will save the results on different gpus to `TMPDIR` and collect them by the rank 0 worker.
+- `TMPDIR`: Temporary directory used for collecting results from multiple workers, available when `--gpu-collect` is not specified.
+- `OPTIONS`: Custom options used for evaluation. Allowed values depend on the arguments of the `evaluate` function in dataset.
+- `AVG_TYPE`: Items to average the test clips. If set to `prob`, it will apply softmax before averaging the clip scores. Otherwise, it will directly average the clip scores.
+- `JOB_LAUNCHER`: Items for distributed job initialization launcher. Allowed choices are `none`, `pytorch`, `slurm`, `mpi`. Especially, if set to none, it will test in a non-distributed mode.
+- `LOCAL_RANK`: ID for local rank. If not specified, it will be set to 0.
+
+Examples:
+
+Assume that you have already downloaded the checkpoints to the directory `checkpoints/`.
+
+1. Test TSN on Kinetics-400 (without saving the test results) and evaluate the top-k accuracy and mean class accuracy.
+
+ ```shell
+ python tools/test.py configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth \
+ --eval top_k_accuracy mean_class_accuracy
+ ```
+
+2. Test TSN on Something-Something V1 with 8 GPUS, and evaluate the top-k accuracy.
+
+ ```shell
+ ./tools/dist_test.sh configs/recognition/tsn/tsn_r50_1x1x8_50e_sthv1_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth \
+ 8 --out results.pkl --eval top_k_accuracy
+ ```
+
+3. Test TSN on Kinetics-400 in slurm environment and evaluate the top-k accuracy
+
+ ```shell
+ python tools/test.py configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py \
+ checkpoints/SOME_CHECKPOINT.pth \
+ --launcher slurm --eval top_k_accuracy
+ ```
+
+
+### High-level APIs for testing a video and rawframes.
+
+Here is an example of building the model and testing a given video.
+
+```python
+import torch
+
+from mmaction.apis import init_recognizer, inference_recognizer
+
+config_file = 'configs/recognition/tsn/tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py'
+# download the checkpoint from model zoo and put it in `checkpoints/`
+checkpoint_file = 'checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth'
+
+# assign the desired device.
+device = 'cuda:0' # or 'cpu'
+device = torch.device(device)
+
+ # build the model from a config file and a checkpoint file
+model = init_recognizer(config_file, checkpoint_file, device=device)
+
+# test a single video and show the result:
+video = 'demo/demo.mp4'
+labels = 'demo/label_map.txt'
+results = inference_recognizer(model, video, labels)
+
+# show the results
+print(f'The top-5 labels with corresponding scores are:')
+for result in results:
+ print(f'{result[0]}: ', result[1])
+```
+
+Here is an example of building the model and testing with a given rawframes directory.
+
+```python
+import torch
+
+from mmaction.apis import init_recognizer, inference_recognizer
+
+config_file = 'configs/recognition/tsn/tsn_r50_inference_1x1x3_100e_kinetics400_rgb.py'
+# download the checkpoint from model zoo and put it in `checkpoints/`
+checkpoint_file = 'checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth'
+
+# assign the desired device.
+device = 'cuda:0' # or 'cpu'
+device = torch.device(device)
+
+ # build the model from a config file and a checkpoint file
+model = init_recognizer(config_file, checkpoint_file, device=device, use_frames=True)
+
+# test rawframe directory of a single video and show the result:
+video = 'SOME_DIR_PATH/'
+labels = 'demo/label_map.txt'
+results = inference_recognizer(model, video, labels, use_frames=True)
+
+# show the results
+print(f'The top-5 labels with corresponding scores are:')
+for result in results:
+ print(f'{result[0]}: ', result[1])
+```
+
+Here is an example of building the model and testing with a given video url.
+
+```python
+import torch
+
+from mmaction.apis import init_recognizer, inference_recognizer
+
+config_file = 'configs/recognition/tsn/tsn_r50_inference_1x1x3_100e_kinetics400_rgb.py'
+# download the checkpoint from model zoo and put it in `checkpoints/`
+checkpoint_file = 'checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth'
+
+# assign the desired device.
+device = 'cuda:0' # or 'cpu'
+device = torch.device(device)
+
+ # build the model from a config file and a checkpoint file
+model = init_recognizer(config_file, checkpoint_file, device=device, use_frames=True)
+
+# test rawframe directory of a single video and show the result:
+video = 'https://www.learningcontainer.com/wp-content/uploads/2020/05/sample-mp4-file.mp4'
+labels = 'demo/label_map.txt'
+results = inference_recognizer(model, video, labels, use_frames=True)
+
+# show the results
+print(f'The top-5 labels with corresponding scores are:')
+for result in results:
+ print(f'{result[0]}: ', result[1])
+```
+
+**Note**: We define `data_prefix` in config files and set it None as default for our provided inference configs.
+If the `data_prefix` is not None, the path for the video file (or rawframe directory) to get will be `data_prefix/video`.
+Here, the `video` is the param in the demo scripts above.
+This detail can be found in `rawframe_dataset.py` and `video_dataset.py`. For example,
+
+* When video (rawframes) path is `SOME_DIR_PATH/VIDEO.mp4` (`SOME_DIR_PATH/VIDEO_NAME/img_xxxxx.jpg`), and `data_prefix` is None in the config file,
+the param `video` should be `SOME_DIR_PATH/VIDEO.mp4` (`SOME_DIR_PATH/VIDEO_NAME`).
+
+* When video (rawframes) path is `SOME_DIR_PATH/VIDEO.mp4` (`SOME_DIR_PATH/VIDEO_NAME/img_xxxxx.jpg`), and `data_prefix` is `SOME_DIR_PATH` in the config file,
+the param `video` should be `VIDEO.mp4` (`VIDEO_NAME`).
+
+* When rawframes path is `VIDEO_NAME/img_xxxxx.jpg`, and `data_prefix` is None in the config file, the param `video` should be `VIDEO_NAME`.
+
+* When passing a url instead of a local video file, you need to use OpenCV as the video decoding backend.
+
+A notebook demo can be found in [demo/demo.ipynb](/demo/demo.ipynb)
+
+
+## Build a Model
+
+### Build a model with basic components
+
+In MMAction2, model components are basically categorized as 4 types.
+
+- recognizer: the whole recognizer model pipeline, usually contains a backbone and cls_head.
+- backbone: usually an FCN network to extract feature maps, e.g., ResNet, BNInception.
+- cls_head: the component for classification task, usually contains an FC layer with some pooling layers.
+- localizer: the model for localization task, currently available: BSN, BMN.
+
+Following some basic pipelines (e.g., `Recognizer2D`), the model structure
+can be customized through config files with no pains.
+
+If we want to implement some new components, e.g., the temporal shift backbone structure as
+in [TSM: Temporal Shift Module for Efficient Video Understanding](https://arxiv.org/abs/1811.08383), there are several things to do.
+
+1. create a new file in `mmaction/models/backbones/resnet_tsm.py`.
+
+ ```python
+ from ..registry import BACKBONES
+ from .resnet import ResNet
+
+ @BACKBONES.register_module()
+ class ResNetTSM(ResNet):
+
+ def __init__(self,
+ depth,
+ num_segments=8,
+ is_shift=True,
+ shift_div=8,
+ shift_place='blockres',
+ temporal_pool=False,
+ **kwargs):
+ pass
+
+ def forward(self, x):
+ # implementation is ignored
+ pass
+ ```
+
+2. Import the module in `mmaction/models/backbones/__init__.py`
+ ```python
+ from .resnet_tsm import ResNetTSM
+ ```
+
+3. modify the config file from
+
+ ```python
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False)
+ ```
+ to
+ ```python
+ backbone=dict(
+ type='ResNetTSM',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False,
+ shift_div=8)
+ ```
+
+### Write a new model
+
+To write a new recognition pipeline, you need to inherit from `BaseRecognizer`,
+which defines the following abstract methods.
+
+- `forward_train()`: forward method of the training mode.
+- `forward_test()`: forward method of the testing mode.
+
+[Recognizer2D](/mmaction/models/recognizers/recognizer2d.py) and [Recognizer3D](/mmaction/models/recognizers/recognizer3d.py)
+are good examples which show how to do that.
+
+
+## Train a Model
+
+### Iteration pipeline
+
+MMAction2 implements distributed training and non-distributed training,
+which uses `MMDistributedDataParallel` and `MMDataParallel` respectively.
+
+We adopt distributed training for both single machine and multiple machines.
+Supposing that the server has 8 GPUs, 8 processes will be started and each process runs on a single GPU.
+
+Each process keeps an isolated model, data loader, and optimizer.
+Model parameters are only synchronized once at the beginning.
+After a forward and backward pass, gradients will be allreduced among all GPUs,
+and the optimizer will update model parameters.
+Since the gradients are allreduced, the model parameter stays the same for all processes after the iteration.
+
+### Training setting
+
+All outputs (log files and checkpoints) will be saved to the working directory,
+which is specified by `work_dir` in the config file.
+
+By default we evaluate the model on the validation set after each epoch, you can change the evaluation interval by modifying the interval argument in the training config
+```python
+evaluation = dict(interval=5) # This evaluate the model per 5 epoch.
+```
+
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you need to set the learning rate proportional to the batch size if you use different GPUs or videos per GPU, e.g., lr=0.01 for 4 GPUs * 2 video/gpu and lr=0.08 for 16 GPUs * 4 video/gpu.
+
+### Train with a single GPU
+
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+If you want to specify the working directory in the command, you can add an argument `--work-dir ${YOUR_WORK_DIR}`.
+
+### Train with multiple GPUs
+
+```shell
+./tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM} [optional arguments]
+```
+
+Optional arguments are:
+
+- `--validate` (**strongly recommended**): Perform evaluation at every k (default value is 5, which can be modified by changing the `interval` value in `evaluation` dict in each config file) epochs during the training.
+- `--work-dir ${WORK_DIR}`: Override the working directory specified in the config file.
+- `--resume-from ${CHECKPOINT_FILE}`: Resume from a previous checkpoint file.
+- `--gpus ${GPU_NUM}`: Number of gpus to use, which is only applicable to non-distributed training.
+- `--gpu-ids ${GPU_IDS}`: IDs of gpus to use, which is only applicable to non-distributed training.
+- `--seed ${SEED}`: Seed id for random state in python, numpy and pytorch to generate random numbers.
+- `--deterministic`: If specified, it will set deterministic options for CUDNN backend.
+- `JOB_LAUNCHER`: Items for distributed job initialization launcher. Allowed choices are `none`, `pytorch`, `slurm`, `mpi`. Especially, if set to none, it will test in a non-distributed mode.
+- `LOCAL_RANK`: ID for local rank. If not specified, it will be set to 0.
+
+Difference between `resume-from` and `load-from`:
+`resume-from` loads both the model weights and optimizer status, and the epoch is also inherited from the specified checkpoint. It is usually used for resuming the training process that is interrupted accidentally.
+`load-from` only loads the model weights and the training epoch starts from 0. It is usually used for finetuning.
+
+Here is an example of using 8 GPUs to load TSN checkpoint.
+
+```shell
+./tools/dist_train.sh configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py 8 --resume-from work_dirs/tsn_r50_1x1x3_100e_kinetics400_rgb/latest.pth
+```
+
+### Train with multiple machines
+
+If you can run MMAction2 on a cluster managed with [slurm](https://slurm.schedmd.com/), you can use the script `slurm_train.sh`. (This script also supports single machine training.)
+
+```shell
+[GPUS=${GPUS}] ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} ${CONFIG_FILE} [--work-dir ${WORK_DIR}]
+```
+
+Here is an example of using 16 GPUs to train TSN on the dev partition in a slurm cluster. (use `GPUS_PER_NODE=8` to specify a single slurm cluster node with 8 GPUs.)
+
+```shell
+GPUS=16 ./tools/slurm_train.sh dev tsn_r50_k400 configs/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb.py --work-dir work_dirs/tsn_r50_1x1x3_100e_kinetics400_rgb
+```
+
+You can check [slurm_train.sh](/tools/slurm_train.sh) for full arguments and environment variables.
+
+If you have just multiple machines connected with ethernet, you can refer to
+pytorch [launch utility](https://pytorch.org/docs/stable/distributed.html#launch-utility).
+Usually it is slow if you do not have high speed networking like InfiniBand.
+
+### Launch multiple jobs on a single machine
+
+If you launch multiple jobs on a single machine, e.g., 2 jobs of 4-GPU training on a machine with 8 GPUs,
+you need to specify different ports (29500 by default) for each job to avoid communication conflict.
+
+If you use `dist_train.sh` to launch training jobs, you can set the port in commands.
+
+```shell
+CUDA_VISIBLE_DEVICES=0,1,2,3 PORT=29500 ./tools/dist_train.sh ${CONFIG_FILE} 4
+CUDA_VISIBLE_DEVICES=4,5,6,7 PORT=29501 ./tools/dist_train.sh ${CONFIG_FILE} 4
+```
+
+If you use launch training jobs with slurm, you need to modify the config files (usually the 6th line from the bottom in config files) to set different communication ports.
+
+In `config1.py`,
+```python
+dist_params = dict(backend='nccl', port=29500)
+```
+
+In `config2.py`,
+```python
+dist_params = dict(backend='nccl', port=29501)
+```
+
+Then you can launch two jobs with `config1.py` ang `config2.py`.
+
+```shell
+CUDA_VISIBLE_DEVICES=0,1,2,3 GPUS=4 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config1.py [--work-dir ${WORK_DIR}]
+CUDA_VISIBLE_DEVICES=4,5,6,7 GPUS=4 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config2.py [--work-dir ${WORK_DIR}]
+```
+
+## Tutorials
+
+Currently, we provide some tutorials for users to [finetune model](tutorials/finetune.md),
+[add new dataset](tutorials/new_dataset.md), [customize data pipelines](tutorials/data_pipeline.md),
+[add new modules](tutorials/new_modules.md), [export a model to ONNX](tutorials/export_model.md) and [customize runtime settings](tutorials/customize_runtime.md).
diff --git a/Downstream/Open-Set-Action-Recognition/docs/imgs/acc_curve.png b/Downstream/Open-Set-Action-Recognition/docs/imgs/acc_curve.png
new file mode 100644
index 0000000..27a2f08
Binary files /dev/null and b/Downstream/Open-Set-Action-Recognition/docs/imgs/acc_curve.png differ
diff --git a/Downstream/Open-Set-Action-Recognition/docs/imgs/data_pipeline.png b/Downstream/Open-Set-Action-Recognition/docs/imgs/data_pipeline.png
new file mode 100644
index 0000000..c5217b1
Binary files /dev/null and b/Downstream/Open-Set-Action-Recognition/docs/imgs/data_pipeline.png differ
diff --git a/Downstream/Open-Set-Action-Recognition/docs/imgs/mmaction2_logo.png b/Downstream/Open-Set-Action-Recognition/docs/imgs/mmaction2_logo.png
new file mode 100644
index 0000000..f0c759b
Binary files /dev/null and b/Downstream/Open-Set-Action-Recognition/docs/imgs/mmaction2_logo.png differ
diff --git a/Downstream/Open-Set-Action-Recognition/docs/imgs/mmaction2_overview.gif b/Downstream/Open-Set-Action-Recognition/docs/imgs/mmaction2_overview.gif
new file mode 100644
index 0000000..9e77ed8
Binary files /dev/null and b/Downstream/Open-Set-Action-Recognition/docs/imgs/mmaction2_overview.gif differ
diff --git a/Downstream/Open-Set-Action-Recognition/docs/index.rst b/Downstream/Open-Set-Action-Recognition/docs/index.rst
new file mode 100644
index 0000000..39bfd1d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/index.rst
@@ -0,0 +1,63 @@
+Welcome to MMAction2's documentation!
+=====================================
+
+.. toctree::
+ :maxdepth: 2
+
+ install.md
+ getting_started.md
+ demo.md
+ benchmark.md
+ config.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Datasets
+
+ data_preparation.md
+ supported_datasets.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Model Zoo
+
+ modelzoo.md
+ recognition_models.md
+ localization_models.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Tutorials
+
+ tutorials/finetune.md
+ tutorials/new_dataset.md
+ tutorials/data_pipeline.md
+ tutorials/new_modules.md
+ tutorials/export_model.md
+ tutorials/customize_runtime.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Useful Tools and Scripts
+
+ useful_tools.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Notes
+
+ changelog.md
+ faq.md
+
+.. toctree::
+ :caption: API Reference
+
+ api.rst
+
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`search`
diff --git a/Downstream/Open-Set-Action-Recognition/docs/install.md b/Downstream/Open-Set-Action-Recognition/docs/install.md
new file mode 100644
index 0000000..fb3a889
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/install.md
@@ -0,0 +1,218 @@
+## Installation
+
+We provide some tips for MMAction2 installation in this file.
+
+
+
+- [Requirements](#requirements)
+- [Install MMAction2](#install-mmaction2)
+- [Install with CPU only](#install-with-cpu-only)
+- [Another option: Docker Image](#another-option--docker-image)
+- [A from-scratch setup script](#a-from-scratch-setup-script)
+- [Developing with multiple MMAction2 versions](#developing-with-multiple-mmaction2-versions)
+- [Verification](#verification)
+
+
+
+### Requirements
+
+- Linux (Windows is not officially supported)
+- Python 3.6+
+- PyTorch 1.3+
+- CUDA 9.2+ (If you build PyTorch from source, CUDA 9.0 is also compatible)
+- GCC 5+
+- [mmcv](https://github.com/open-mmlab/mmcv) 1.1.1+
+- Numpy
+- ffmpeg (4.2 is preferred)
+- [decord](https://github.com/dmlc/decord) (optional, 0.4.1+): Install CPU version by `pip install decord==0.4.1` and install GPU version from source
+- [PyAV](https://github.com/mikeboers/PyAV) (optional): `conda install av -c conda-forge -y`
+- [PyTurboJPEG](https://github.com/lilohuang/PyTurboJPEG) (optional): `pip install PyTurboJPEG`
+- [denseflow](https://github.com/open-mmlab/denseflow) (optional): See [here](https://github.com/innerlee/setup) for simple install scripts.
+- [moviepy](https://zulko.github.io/moviepy/) (optional): `pip install moviepy`. See [here](https://zulko.github.io/moviepy/install.html) for official installation. **Note**(according to [this issue](https://github.com/Zulko/moviepy/issues/693)) that:
+ 1. For Windows users, [ImageMagick](https://www.imagemagick.org/script/index.php) will not be automatically detected by MoviePy,
+ there is a need to modify `moviepy/config_defaults.py` file by providing the path to the ImageMagick binary called `magick`, like `IMAGEMAGICK_BINARY = "C:\\Program Files\\ImageMagick_VERSION\\magick.exe"`
+ 2. For Linux users, there is a need to modify the `/etc/ImageMagick-6/policy.xml` file by commenting out
+ ` ` to ``, if [ImageMagick](https://www.imagemagick.org/script/index.php) is not detected by `moviepy`.
+- [Pillow-SIMD](https://docs.fast.ai/performance.html#pillow-simd) (optional): Install it by the following scripts.
+```shell
+conda uninstall -y --force pillow pil jpeg libtiff libjpeg-turbo
+pip uninstall -y pillow pil jpeg libtiff libjpeg-turbo
+conda install -yc conda-forge libjpeg-turbo
+CFLAGS="${CFLAGS} -mavx2" pip install --upgrade --no-cache-dir --force-reinstall --no-binary :all: --compile pillow-simd
+conda install -y jpeg libtiff
+```
+
+**Note**: You need to run `pip uninstall mmcv` first if you have mmcv installed.
+If mmcv and mmcv-full are both installed, there will be `ModuleNotFoundError`.
+
+### Install MMAction2
+
+a. Create a conda virtual environment and activate it.
+
+```shell
+conda create -n open-mmlab python=3.7 -y
+conda activate open-mmlab
+```
+
+b. Install PyTorch and torchvision following the [official instructions](https://pytorch.org/), e.g.,
+
+```shell
+conda install pytorch torchvision -c pytorch
+```
+
+Note: Make sure that your compilation CUDA version and runtime CUDA version match.
+You can check the supported CUDA version for precompiled packages on the [PyTorch website](https://pytorch.org/).
+
+`E.g.1` If you have CUDA 10.1 installed under `/usr/local/cuda` and would like to install PyTorch 1.5,
+you need to install the prebuilt PyTorch with CUDA 10.1.
+
+```shell
+conda install pytorch cudatoolkit=10.1 torchvision -c pytorch
+```
+
+`E.g.2` If you have CUDA 9.2 installed under `/usr/local/cuda` and would like to install PyTorch 1.3.1.,
+you need to install the prebuilt PyTorch with CUDA 9.2.
+
+```shell
+conda install pytorch=1.3.1 cudatoolkit=9.2 torchvision=0.4.2 -c pytorch
+```
+
+If you build PyTorch from source instead of installing the prebuilt package, you can use more CUDA versions such as 9.0.
+
+c. Install mmcv, we recommend you to install the pre-build mmcv as below.
+
+```shell
+pip install mmcv-full==latest+torch1.5.0+cu101 -f https://download.openmmlab.com/mmcv/dist/index.html
+```
+
+See [here](https://github.com/open-mmlab/mmcv#install-with-pip) for different versions of MMCV compatible to different PyTorch and CUDA versions.
+Optionally you can choose to compile mmcv from source by the following command
+
+```shell
+git clone https://github.com/open-mmlab/mmcv.git
+cd mmcv
+MMCV_WITH_OPS=1 pip install -e . # package mmcv-full, which contains cuda ops, will be installed after this step
+# OR pip install -e . # package mmcv, which contains no cuda ops, will be installed after this step
+cd ..
+```
+
+Or directly run
+
+```shell
+pip install mmcv-full
+# alternative: pip install mmcv
+```
+
+**Important:** You need to run `pip uninstall mmcv` first if you have mmcv installed. If mmcv and mmcv-full are both installed, there will be `ModuleNotFoundError`.
+
+d. Clone the MMAction2 repository
+
+```shell
+git clone https://github.com/open-mmlab/mmaction2.git
+cd mmaction2
+```
+
+d. Install build requirements and then install MMAction2.
+
+```shell
+pip install -r requirements/build.txt
+pip install -v -e . # or "python setup.py develop"
+```
+
+If you build MMAction2 on macOS, replace the last command with
+
+```
+CC=clang CXX=clang++ CFLAGS='-stdlib=libc++' pip install -e .
+```
+
+
+Note:
+
+1. The git commit id will be written to the version number with step d, e.g. 0.6.0+2e7045c. The version will also be saved in trained models.
+It is recommended that you run step d each time you pull some updates from github. If C++/CUDA codes are modified, then this step is compulsory.
+
+2. Following the above instructions, MMAction2 is installed on `dev` mode, any local modifications made to the code will take effect without the need to reinstall it (unless you submit some commits and want to update the version number).
+
+3. If you would like to use `opencv-python-headless` instead of `opencv-python`,
+you can install it before installing MMCV.
+
+4. If you would like to use `PyAV`, you can install it with `conda install av -c conda-forge -y`.
+
+5. Some dependencies are optional. Running `python setup.py develop` will only install the minimum runtime requirements.
+To use optional dependencies like `decord`, either install them with `pip install -r requirements/optional.txt`
+or specify desired extras when calling `pip` (e.g. `pip install -v -e .[optional]`,
+valid keys for the `[optional]` field are `all`, `tests`, `build`, and `optional`) like `pip install -v -e .[tests,build]`.
+
+### Install with CPU only
+The code can be built for CPU only environment (where CUDA isn't available).
+
+In CPU mode you can run the demo/demo.py for example.
+
+### Another option: Docker Image
+
+We provide a [Dockerfile](/docker/Dockerfile) to build an image.
+
+```shell
+# build an image with PyTorch 1.6.0, CUDA 10.1, CUDNN 7.
+docker build -f ./docker/Dockerfile --rm -t mmaction2 .
+```
+
+**Important:** Make sure you've installed the [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker).
+
+Run it with command:
+
+```shell
+docker run --gpus all --shm-size=8g -it -v {DATA_DIR}:/mmaction2/data mmaction2
+```
+
+### A from-scratch setup script
+
+Here is a full script for setting up MMAction2 with conda and link the dataset path (supposing that your Kinetics-400 dataset path is $KINETICS400_ROOT).
+
+```shell
+conda create -n open-mmlab python=3.7 -y
+conda activate open-mmlab
+
+# install latest pytorch prebuilt with the default prebuilt CUDA version (usually the latest)
+conda install -c pytorch pytorch torchvision -y
+
+# install the latest mmcv or mmcv-full, here we take mmcv as example
+pip install mmcv
+
+# install mmaction2
+git clone https://github.com/open-mmlab/mmaction2.git
+cd mmaction2
+pip install -r requirements/build.txt
+python setup.py develop
+
+mkdir data
+ln -s $KINETICS400_ROOT data
+```
+
+### Developing with multiple MMAction2 versions
+
+The train and test scripts already modify the `PYTHONPATH` to ensure the script use the MMAction2 in the current directory.
+
+To use the default MMAction2 installed in the environment rather than that you are working with, you can remove the following line in those scripts.
+
+```shell
+PYTHONPATH="$(dirname $0)/..":$PYTHONPATH
+```
+
+### Verification
+
+To verify whether MMAction2 and the required environment are installed correctly,
+we can run sample python codes to initialize a recognizer and inference a demo video:
+
+```python
+import torch
+from mmaction.apis import init_recognizer, inference_recognizer
+
+config_file = 'configs/recognition/tsn/tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py'
+device = 'cuda:0' # or 'cpu'
+device = torch.device(device)
+
+model = init_recognizer(config_file, device=device)
+# inference the demo video
+inference_recognizer(model, 'demo/demo.mp4', 'demo/label_map.txt')
+```
diff --git a/Downstream/Open-Set-Action-Recognition/docs/make.bat b/Downstream/Open-Set-Action-Recognition/docs/make.bat
new file mode 100644
index 0000000..922152e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/make.bat
@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+ set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=.
+set BUILDDIR=_build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+ echo.
+ echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+ echo.installed, then set the SPHINXBUILD environment variable to point
+ echo.to the full path of the 'sphinx-build' executable. Alternatively you
+ echo.may add the Sphinx directory to PATH.
+ echo.
+ echo.If you don't have Sphinx installed, grab it from
+ echo.http://sphinx-doc.org/
+ exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd
diff --git a/Downstream/Open-Set-Action-Recognition/docs/merge_docs.sh b/Downstream/Open-Set-Action-Recognition/docs/merge_docs.sh
new file mode 100644
index 0000000..957e8be
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/merge_docs.sh
@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+
+sed -i '$a\\n' ../demo/README.md
+
+sed -i 's/(\/tools\/data\/activitynet\/README.md/(#activitynet/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/kinetics\/README.md/(#kinetics-400-600-700/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/mit\/README.md/(#moments-in-time/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/mmit\/README.md/(#multi-moments-in-time/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/sthv1\/README.md/(#something-something-v1/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/sthv2\/README.md/(#something-something-v2/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/thumos14\/README.md/(#thumos-14/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/ucf101\/README.md/(#ucf-101/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/ucf101_24\/README.md/(#ucf101-24/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/jhmdb\/README.md/(#jhmdb/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/hvu\/README.md/(#hvu/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/hmdb51\/README.md/(#hmdb51/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/jester\/README.md/(#jester/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/ava\/README.md/(#ava/g' supported_datasets.md
+sed -i 's/(\/tools\/data\/gym\/README.md/(#gym/g' supported_datasets.md
+
+cat ../configs/localization/*/*.md > localization_models.md
+cat ../configs/recognition/*/*.md > recognition_models.md
+cat ../configs/recognition_audio/*/*.md >> recognition_models.md
+cat ../tools/data/*/*.md > prepare_data.md
+cat ../demo/README.md > demo.md
+
+sed -i 's/#/##&/' localization_models.md
+sed -i 's/#/##&/' recognition_models.md
+sed -i 's/md###t/html#t/g' localization_models.md
+sed -i 's/md###t/html#t/g' recognition_models.md
+sed -i "s/md###t/html#t/g" demo.md
+
+sed -i 's/# Preparing/# /g' prepare_data.md
+sed -i 's/#/##&/' prepare_data.md
+
+sed -i '1i\## Action Localization Models' localization_models.md
+sed -i '1i\## Action Recognition Models' recognition_models.md
+
+cat prepare_data.md >> supported_datasets.md
+
+sed -i 's/](\/docs\//](/g' recognition_models.md # remove /docs/ for link used in doc site
+sed -i 's/](\/docs\//](/g' localization_models.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmaction2/tree/master/=g' recognition_models.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmaction2/tree/master/=g' localization_models.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmaction2/tree/master/=g' benchmark.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmaction2/tree/master/=g' getting_started.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmaction2/tree/master/=g' install.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmaction2/tree/master/=g' config.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmaction2/tree/master/=g' changelog.md
+sed -i 's/](\/docs\//](/g' ./tutorials/*.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmaction2/tree/master/=g' ./tutorials/*.md
+sed -i 's/](\/docs\//](/g' supported_datasets.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmaction2/tree/master/=g' supported_datasets.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmaction2/tree/master/=g' demo.md
diff --git a/Downstream/Open-Set-Action-Recognition/docs/stat.py b/Downstream/Open-Set-Action-Recognition/docs/stat.py
new file mode 100644
index 0000000..8555089
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/stat.py
@@ -0,0 +1,59 @@
+#!/usr/bin/env python
+import functools as func
+import glob
+import re
+
+files = sorted(glob.glob('*_models.md'))
+
+stats = []
+
+for f in files:
+ with open(f, 'r') as content_file:
+ content = content_file.read()
+
+ # title
+ title = content.split('\n')[0].replace('#', '')
+
+ # count papers
+ papers = set(x.lower().strip()
+ for x in re.findall(r'\btitle={(.*)}', content))
+ paperlist = '\n'.join(sorted(' - ' + x for x in papers))
+
+ # count configs
+ configs = set(x.lower().strip()
+ for x in re.findall(r'https.*configs/.*\.py', content))
+
+ # count ckpts
+ ckpts = set(x.lower().strip()
+ for x in re.findall(r'https://download.*\.pth', content)
+ if 'mmaction' in x)
+
+ statsmsg = f"""
+## [{title}]({f})
+
+* Number of checkpoints: {len(ckpts)}
+* Number of configs: {len(configs)}
+* Number of papers: {len(papers)}
+{paperlist}
+
+ """
+
+ stats.append((papers, configs, ckpts, statsmsg))
+
+allpapers = func.reduce(lambda a, b: a.union(b), [p for p, _, _, _ in stats])
+allconfigs = func.reduce(lambda a, b: a.union(b), [c for _, c, _, _ in stats])
+allckpts = func.reduce(lambda a, b: a.union(b), [c for _, _, c, _ in stats])
+msglist = '\n'.join(x for _, _, _, x in stats)
+
+modelzoo = f"""
+# Model Zoo Statistics
+
+* Number of checkpoints: {len(allckpts)}
+* Number of configs: {len(allconfigs)}
+* Number of papers: {len(allpapers)}
+
+{msglist}
+"""
+
+with open('modelzoo.md', 'w') as f:
+ f.write(modelzoo)
diff --git a/Downstream/Open-Set-Action-Recognition/docs/supported_datasets.md b/Downstream/Open-Set-Action-Recognition/docs/supported_datasets.md
new file mode 100644
index 0000000..1caccee
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/supported_datasets.md
@@ -0,0 +1,21 @@
+# Supported Datasets
+
+- [UCF101](https://www.crcv.ucf.edu/research/data-sets/ucf101/): See [preparing_ucf101](/tools/data/ucf101/README.md).
+- [HMDB51](https://serre-lab.clps.brown.edu/resource/hmdb-a-large-human-motion-database/): See [preparing_hmdb51](/tools/data/hmdb51/README.md).
+- [Kinetics-[400/600/700]](https://deepmind.com/research/open-source/kinetics): See [preparing_kinetics](/tools/data/kinetics/README.md)
+- [THUMOS14](https://www.crcv.ucf.edu/THUMOS14/download.html): See [preparing_thumos14](/tools/data/thumos14/README.md)
+- [Something-Something V1](https://20bn.com/datasets/something-something/v1): See [preparing_sthv1](/tools/data/sthv1/README.md)
+- [Something-Something V2](https://20bn.com/datasets/something-something): See [preparing_sthv2](/tools/data/sthv2/README.md)
+- [Moments in Time](http://moments.csail.mit.edu/): See [preparing_mit](/tools/data/mit/README.md)
+- [Multi-Moments in Time](http://moments.csail.mit.edu/challenge_iccv_2019.html): See [preparing_mmit](/tools/data/mmit/README.md)
+- [ActivityNet](http://activity-net.org/): See [praparing_activitynet](/tools/data/activitynet/README.md)
+- [UCF101-24](http://www.thumos.info/download.html): See [preparing_ucf101_24](/tools/data/ucf101_24/README.md)
+- [JHMDB](http://jhmdb.is.tue.mpg.de/): See [preparing_jhmdb](/tools/data/jhmdb/README.md)
+- [HVU](https://github.com/holistic-video-understanding/HVU-Dataset): See [preparing_hvu](/tools/data/hvu/README.md)
+- [Jester](https://20bn.com/datasets/jester/v1): See [preparing_jester](/tools/data/jester/README.md)
+- [AVA](https://research.google.com/ava/index.html): See [preparing_ava](/tools/data/ava/README.md)
+- [GYM](https://sdolivia.github.io/FineGym/): See [preparing_gym](/tools/data/gym/README.md)
+
+The supported datasets are listed above.
+We provide shell scripts for data preparation under the path `$MMACTION2/tools/data/`.
+Below is the detailed tutorials of data deployment for each dataset.
diff --git a/Downstream/Open-Set-Action-Recognition/docs/tutorials/customize_runtime.md b/Downstream/Open-Set-Action-Recognition/docs/tutorials/customize_runtime.md
new file mode 100644
index 0000000..60199cf
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/tutorials/customize_runtime.md
@@ -0,0 +1,353 @@
+# Tutorial 6: Customize Runtime Settings
+
+In this tutorial, we will introduce some methods about how to customize optimization methods, training schedules, workflow and hooks when running your own settings for the project.
+
+
+
+- [Customize Optimization Methods](#customize-optimization-methods)
+ * [Customize optimizer supported by PyTorch](#customize-optimizer-supported-by-pytorch)
+ * [Customize self-implemented optimizer](#customize-self-implemented-optimizer)
+ + [1. Define a new optimizer](#1-define-a-new-optimizer)
+ + [2. Add the optimizer to registry](#2-add-the-optimizer-to-registry)
+ + [3. Specify the optimizer in the config file](#3-specify-the-optimizer-in-the-config-file)
+ * [Customize optimizer constructor](#customize-optimizer-constructor)
+ * [Additional settings](#additional-settings)
+- [Customize Training Schedules](#customize-training-schedules)
+- [Customize Workflow](#customize-workflow)
+- [Customize Hooks](#customize-hooks)
+ * [Customize self-implemented hooks](#customize-self-implemented-hooks)
+ + [1. Implement a new hook](#1-implement-a-new-hook)
+ + [2. Register the new hook](#2-register-the-new-hook)
+ + [3. Modify the config](#3-modify-the-config)
+ * [Use hooks implemented in MMCV](#use-hooks-implemented-in-mmcv)
+ * [Modify default runtime hooks](#modify-default-runtime-hooks)
+ + [Checkpoint config](#checkpoint-config)
+ + [Log config](#log-config)
+ + [Evaluation config](#evaluation-config)
+
+
+
+## Customize Optimization Methods
+
+### Customize optimizer supported by PyTorch
+
+We already support to use all the optimizers implemented by PyTorch, and the only modification is to change the `optimizer` field of config files.
+For example, if you want to use `Adam`, the modification could be as the following.
+
+```python
+optimizer = dict(type='Adam', lr=0.0003, weight_decay=0.0001)
+```
+
+To modify the learning rate of the model, the users only need to modify the `lr` in the config of optimizer.
+The users can directly set arguments following the [API doc](https://pytorch.org/docs/stable/optim.html?highlight=optim#module-torch.optim) of PyTorch.
+
+For example, if you want to use `Adam` with the setting like `torch.optim.Adam(parms, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False)` in PyTorch,
+the modification could be set as the following.
+
+```python
+optimizer = dict(type='Adam', lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False)
+```
+
+### Customize self-implemented optimizer
+
+#### 1. Define a new optimizer
+
+A customized optimizer could be defined as following.
+
+Assume you want to add an optimizer named `MyOptimizer`, which has arguments `a`, `b`, and `c`.
+You need to create a new directory named `mmaction/core/optimizer`.
+And then implement the new optimizer in a file, e.g., in `mmaction/core/optimizer/my_optimizer.py`:
+
+```python
+from .registry import OPTIMIZERS
+from torch.optim import Optimizer
+
+
+@OPTIMIZERS.register_module()
+class MyOptimizer(Optimizer):
+
+ def __init__(self, a, b, c):
+
+```
+
+#### 2. Add the optimizer to registry
+
+To find the above module defined above, this module should be imported into the main namespace at first. There are two ways to achieve it.
+
+- Modify `mmaction/core/optimizer/__init__.py` to import it.
+
+ The newly defined module should be imported in `mmaction/core/optimizer/__init__.py` so that the registry will
+ find the new module and add it:
+
+```python
+from .my_optimizer import MyOptimizer
+```
+
+- Use `custom_imports` in the config to manually import it
+
+```python
+custom_imports = dict(imports=['mmaction.core.optimizer.my_optimizer'], allow_failed_imports=False)
+```
+
+The module `mmaction.core.optimizer.my_optimizer` will be imported at the beginning of the program and the class `MyOptimizer` is then automatically registered.
+Note that only the package containing the class `MyOptimizer` should be imported. `mmaction.core.optimizer.my_optimizer.MyOptimizer` **cannot** be imported directly.
+
+#### 3. Specify the optimizer in the config file
+
+Then you can use `MyOptimizer` in `optimizer` field of config files.
+In the configs, the optimizers are defined by the field `optimizer` like the following:
+
+```python
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+```
+
+To use your own optimizer, the field can be changed to
+
+```python
+optimizer = dict(type='MyOptimizer', a=a_value, b=b_value, c=c_value)
+```
+
+### Customize optimizer constructor
+
+Some models may have some parameter-specific settings for optimization, e.g. weight decay for BatchNorm layers.
+The users can do those fine-grained parameter tuning through customizing optimizer constructor.
+
+```python
+from mmcv.utils import build_from_cfg
+
+from mmcv.runner.optimizer import OPTIMIZER_BUILDERS, OPTIMIZERS
+from mmaction.utils import get_root_logger
+from .my_optimizer import MyOptimizer
+
+
+@OPTIMIZER_BUILDERS.register_module()
+class MyOptimizerConstructor:
+
+ def __init__(self, optimizer_cfg, paramwise_cfg=None):
+ pass
+
+ def __call__(self, model):
+
+ return my_optimizer
+```
+
+The default optimizer constructor is implemented [here](https://github.com/open-mmlab/mmcv/blob/9ecd6b0d5ff9d2172c49a182eaa669e9f27bb8e7/mmcv/runner/optimizer/default_constructor.py#L11),
+which could also serve as a template for new optimizer constructor.
+
+### Additional settings
+
+Tricks not implemented by the optimizer should be implemented through optimizer constructor (e.g., set parameter-wise learning rates) or hooks.
+We list some common settings that could stabilize the training or accelerate the training. Feel free to create PR, issue for more settings.
+
+- __Use gradient clip to stabilize training__:
+ Some models need gradient clip to clip the gradients to stabilize the training process. An example is as below:
+
+ ```python
+ optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
+ ```
+
+- __Use momentum schedule to accelerate model convergence__:
+ We support momentum scheduler to modify model's momentum according to learning rate, which could make the model converge in a faster way.
+ Momentum scheduler is usually used with LR scheduler, for example, the following config is used in 3D detection to accelerate convergence.
+ For more details, please refer to the implementation of [CyclicLrUpdater](https://github.com/open-mmlab/mmcv/blob/f48241a65aebfe07db122e9db320c31b685dc674/mmcv/runner/hooks/lr_updater.py#L327)
+ and [CyclicMomentumUpdater](https://github.com/open-mmlab/mmcv/blob/f48241a65aebfe07db122e9db320c31b685dc674/mmcv/runner/hooks/momentum_updater.py#L130).
+
+ ```python
+ lr_config = dict(
+ policy='cyclic',
+ target_ratio=(10, 1e-4),
+ cyclic_times=1,
+ step_ratio_up=0.4,
+ )
+ momentum_config = dict(
+ policy='cyclic',
+ target_ratio=(0.85 / 0.95, 1),
+ cyclic_times=1,
+ step_ratio_up=0.4,
+ )
+ ```
+
+## Customize Training Schedules
+
+we use step learning rate with default value in config files, this calls [`StepLRHook`](https://github.com/open-mmlab/mmcv/blob/f48241a65aebfe07db122e9db320c31b685dc674/mmcv/runner/hooks/lr_updater.py#L153) in MMCV.
+We support many other learning rate schedule [here](https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/lr_updater.py), such as `CosineAnnealing` and `Poly` schedule. Here are some examples
+
+- Poly schedule:
+
+ ```python
+ lr_config = dict(policy='poly', power=0.9, min_lr=1e-4, by_epoch=False)
+ ```
+
+- ConsineAnnealing schedule:
+
+ ```python
+ lr_config = dict(
+ policy='CosineAnnealing',
+ warmup='linear',
+ warmup_iters=1000,
+ warmup_ratio=1.0 / 10,
+ min_lr_ratio=1e-5)
+ ```
+
+## Customize Workflow
+
+By default, we recommend users to use `EpochEvalHook` to do evaluation after training epoch, but they can still use `val` workflow as an alternative.
+
+Workflow is a list of (phase, epochs) to specify the running order and epochs. By default it is set to be
+
+```python
+workflow = [('train', 1)]
+```
+
+which means running 1 epoch for training.
+Sometimes user may want to check some metrics (e.g. loss, accuracy) about the model on the validate set.
+In such case, we can set the workflow as
+
+```python
+[('train', 1), ('val', 1)]
+```
+
+so that 1 epoch for training and 1 epoch for validation will be run iteratively.
+
+**Note**:
+
+1. The parameters of model will not be updated during val epoch.
+2. Keyword `total_epochs` in the config only controls the number of training epochs and will not affect the validation workflow.
+3. Workflows `[('train', 1), ('val', 1)]` and `[('train', 1)]` will not change the behavior of `EpochEvalHook` because `EpochEvalHook` is called by `after_train_epoch` and validation workflow only affect hooks that are called through `after_val_epoch`.
+Therefore, the only difference between `[('train', 1), ('val', 1)]` and ``[('train', 1)]`` is that the runner will calculate losses on validation set after each training epoch.
+
+## Customize Hooks
+
+### Customize self-implemented hooks
+
+#### 1. Implement a new hook
+
+Here we give an example of creating a new hook in MMAction2 and using it in training.
+
+```python
+from mmcv.runner import HOOKS, Hook
+
+
+@HOOKS.register_module()
+class MyHook(Hook):
+
+ def __init__(self, a, b):
+ pass
+
+ def before_run(self, runner):
+ pass
+
+ def after_run(self, runner):
+ pass
+
+ def before_epoch(self, runner):
+ pass
+
+ def after_epoch(self, runner):
+ pass
+
+ def before_iter(self, runner):
+ pass
+
+ def after_iter(self, runner):
+ pass
+```
+
+Depending on the functionality of the hook, the users need to specify what the hook will do at each stage of the training in `before_run`, `after_run`, `before_epoch`, `after_epoch`, `before_iter`, and `after_iter`.
+
+#### 2. Register the new hook
+
+Then we need to make `MyHook` imported. Assuming the file is in `mmaction/core/utils/my_hook.py` there are two ways to do that:
+
+- Modify `mmaction/core/utils/__init__.py` to import it.
+
+ The newly defined module should be imported in `mmaction/core/utils/__init__.py` so that the registry will
+ find the new module and add it:
+
+```python
+from .my_hook import MyHook
+```
+
+- Use `custom_imports` in the config to manually import it
+
+```python
+custom_imports = dict(imports=['mmaction.core.utils.my_hook'], allow_failed_imports=False)
+```
+
+#### 3. Modify the config
+
+```python
+custom_hooks = [
+ dict(type='MyHook', a=a_value, b=b_value)
+]
+```
+
+You can also set the priority of the hook by adding key `priority` to `'NORMAL'` or `'HIGHEST'` as below
+
+```python
+custom_hooks = [
+ dict(type='MyHook', a=a_value, b=b_value, priority='NORMAL')
+]
+```
+
+By default the hook's priority is set as `NORMAL` during registration.
+
+
+### Use hooks implemented in MMCV
+
+If the hook is already implemented in MMCV, you can directly modify the config to use the hook as below
+
+```python
+mmcv_hooks = [
+ dict(type='MMCVHook', a=a_value, b=b_value, priority='NORMAL')
+]
+```
+
+### Modify default runtime hooks
+
+There are some common hooks that are not registered through `custom_hooks` but has been registered by default when importing MMCV, they are
+
+- log_config
+- checkpoint_config
+- evaluation
+- lr_config
+- optimizer_config
+- momentum_config
+
+In those hooks, only the logger hook has the `VERY_LOW` priority, others' priority are `NORMAL`.
+The above-mentioned tutorials already cover how to modify `optimizer_config`, `momentum_config`, and `lr_config`.
+Here we reveals how what we can do with `log_config`, `checkpoint_config`, and `evaluation`.
+
+#### Checkpoint config
+
+The MMCV runner will use `checkpoint_config` to initialize [`CheckpointHook`](https://github.com/open-mmlab/mmcv/blob/9ecd6b0d5ff9d2172c49a182eaa669e9f27bb8e7/mmcv/runner/hooks/checkpoint.py#L9).
+
+```python
+checkpoint_config = dict(interval=1)
+```
+
+The users could set `max_keep_ckpts` to only save only small number of checkpoints or decide whether to store state dict of optimizer by `save_optimizer`.
+More details of the arguments are [here](https://mmcv.readthedocs.io/en/latest/api.html#mmcv.runner.CheckpointHook)
+
+#### Log config
+
+The `log_config` wraps multiple logger hooks and enables to set intervals. Now MMCV supports `WandbLoggerHook`, `MlflowLoggerHook`, and `TensorboardLoggerHook`.
+The detail usages can be found in the [doc](https://mmcv.readthedocs.io/en/latest/api.html#mmcv.runner.LoggerHook).
+
+```python
+log_config = dict(
+ interval=50,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook')
+ ])
+```
+
+#### Evaluation config
+
+The config of `evaluation` will be used to initialize the [`EpochEvalHook`](https://github.com/open-mmlab/mmaction2/blob/master/mmaction/core/evaluation/eval_hooks.py#L12).
+Except the key `interval`, other arguments such as `metrics` will be passed to the `dataset.evaluate()`
+
+```python
+evaluation = dict(interval=1, metrics='bbox')
+```
diff --git a/Downstream/Open-Set-Action-Recognition/docs/tutorials/data_pipeline.md b/Downstream/Open-Set-Action-Recognition/docs/tutorials/data_pipeline.md
new file mode 100644
index 0000000..d305987
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/tutorials/data_pipeline.md
@@ -0,0 +1,213 @@
+# Tutorial 3: Customize Data Pipelines
+
+In this tutorial, we will introduce some methods about the design of data pipelines, and how to customize and extend your own data pipelines for the project.
+
+
+
+- [Design of Data Pipelines](#design-of-data-pipelines)
+ * [Data loading](#data-loading)
+ * [Pre-processing](#pre-processing)
+ * [Formatting](#formatting)
+- [Extend and Use Custom Pipelines](#extend-and-use-custom-pipelines)
+
+
+
+## Design of Data Pipelines
+
+Following typical conventions, we use `Dataset` and `DataLoader` for data loading
+with multiple workers. `Dataset` returns a dict of data items corresponding
+the arguments of models' forward method.
+Since the data in action recognition & localization may not be the same size (image size, gt bbox size, etc.),
+The `DataContainer` in MMCV is used to help collect and distribute data of different sizes.
+See [here](https://github.com/open-mmlab/mmcv/blob/master/mmcv/parallel/data_container.py) for more details.
+
+The data preparation pipeline and the dataset is decomposed. Usually a dataset
+defines how to process the annotations and a data pipeline defines all the steps to prepare a data dict.
+A pipeline consists of a sequence of operations. Each operation takes a dict as input and also output a dict for the next operation.
+
+We present a typical pipeline in the following figure. The blue blocks are pipeline operations.
+With the pipeline going on, each operator can add new keys (marked as green) to the result dict or update the existing keys (marked as orange).
+![pipeline figure](../imgs/data_pipeline.png)
+
+The operations are categorized into data loading, pre-processing and formatting.
+
+Here is a pipeline example for TSN.
+```python
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)
+train_pipeline = [
+ dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=3),
+ dict(type='RawFrameDecode', io_backend='disk'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(
+ type='MultiScaleCrop',
+ input_size=224,
+ scales=(1, 0.875, 0.75, 0.66),
+ random_crop=False,
+ max_wh_scale_gap=1),
+ dict(type='Resize', scale=(224, 224), keep_ratio=False),
+ dict(type='Flip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+]
+val_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=3,
+ test_mode=True),
+ dict(type='RawFrameDecode', io_backend='disk'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='CenterCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+test_pipeline = [
+ dict(
+ type='SampleFrames',
+ clip_len=1,
+ frame_interval=1,
+ num_clips=25,
+ test_mode=True),
+ dict(type='RawFrameDecode', io_backend='disk'),
+ dict(type='Resize', scale=(-1, 256)),
+ dict(type='TenCrop', crop_size=224),
+ dict(type='Flip', flip_ratio=0),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs'])
+]
+```
+
+For each operation, we list the related dict fields that are added/updated/removed.
+
+### Data loading
+
+`SampleFrames`
+- add: frame_inds, clip_len, frame_interval, num_clips, *total_frames
+
+`DenseSampleFrames`
+- add: frame_inds, clip_len, frame_interval, num_clips, *total_frames
+
+`PyAVDecode`
+- add: imgs, original_shape
+- update: *frame_inds
+
+`DecordDecode`
+- add: imgs, original_shape
+- update: *frame_inds
+
+`OpenCVDecode`
+- add: imgs, original_shape
+- update: *frame_inds
+
+`RawFrameDecode`
+- add: imgs, original_shape
+- update: *frame_inds
+
+### Pre-processing
+
+`RandomCrop`
+- add: crop_bbox, img_shape
+- update: imgs
+
+`RandomResizedCrop`
+- add: crop_bbox, img_shape
+- update: imgs
+
+`MultiScaleCrop`
+- add: crop_bbox, img_shape, scales
+- update: imgs
+
+`Resize`
+- add: img_shape, keep_ratio, scale_factor
+- update: imgs
+
+`Flip`
+- add: flip, flip_direction
+- update: imgs
+
+`Normalize`
+- add: img_norm_cfg
+- update: imgs
+
+`CenterCrop`
+- add: crop_bbox, img_shape
+- update: imgs
+
+`ThreeCrop`
+- add: crop_bbox, img_shape
+- update: imgs
+
+`TenCrop`
+- add: crop_bbox, img_shape
+- update: imgs
+
+`MultiGroupCrop`
+- add: crop_bbox, img_shape
+- update: imgs
+
+### Formatting
+
+`ToTensor`
+- update: specified by `keys`.
+
+`ImageToTensor`
+- update: specified by `keys`.
+
+`Transpose`
+- update: specified by `keys`.
+
+`Collect`
+- add: img_meta (the keys of img_meta is specified by `meta_keys`)
+- remove: all other keys except for those specified by `keys`
+
+It is **noteworthy** that the first key, commonly `imgs`, will be used as the main key to calculate the batch size.
+
+`FormatShape`
+- add: input_shape
+- update: imgs
+
+## Extend and Use Custom Pipelines
+
+1. Write a new pipeline in any file, e.g., `my_pipeline.py`. It takes a dict as input and return a dict.
+
+ ```python
+ from mmaction.datasets import PIPELINES
+
+ @PIPELINES.register_module()
+ class MyTransform:
+
+ def __call__(self, results):
+ results['key'] = value
+ return results
+ ```
+
+2. Import the new class.
+
+ ```python
+ from .my_pipeline import MyTransform
+ ```
+
+3. Use it in config files.
+
+ ```python
+ img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+ train_pipeline = [
+ dict(type='DenseSampleFrames', clip_len=8, frame_interval=8, num_clips=1),
+ dict(type='RawFrameDecode', io_backend='disk'),
+ dict(type='MyTransform'), # use a custom pipeline
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='FormatShape', input_format='NCTHW'),
+ dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),
+ dict(type='ToTensor', keys=['imgs', 'label'])
+ ]
+ ```
diff --git a/Downstream/Open-Set-Action-Recognition/docs/tutorials/export_model.md b/Downstream/Open-Set-Action-Recognition/docs/tutorials/export_model.md
new file mode 100644
index 0000000..9d046e8
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/tutorials/export_model.md
@@ -0,0 +1,62 @@
+# Tutorial 5: Exporting a model to ONNX
+
+Open Neural Network Exchange [(ONNX)](https://onnx.ai/) is an open ecosystem that empowers AI developers to choose the right tools as their project evolves.
+
+
+
+- [Supported Models](#supported-models)
+- [Usage](#usage)
+ * [Prerequisite](#prerequisite)
+ * [Recognizers](#recognizers)
+ * [Localizer](#localizer)
+
+
+
+## Supported Models
+So far, our codebase supports onnx exporting from pytorch models trained with MMAction2. The supported models are:
+
++ I3D
++ TSN
++ TIN
++ TSM
++ R(2+1)D
++ SLOWFAST
++ SLOWONLY
++ BMN
++ BSN(tem, pem)
+
+## Usage
+For simple exporting, you can use the [script](/tools/pytorch2onnx.py) here. Note that the package `onnx` and `onnxruntime` are required for verification after exporting.
+
+### Prerequisite
+First, install onnx.
+```shell
+pip install onnx onnxruntime
+```
+
+We provide a python script to export the pytorch model trained by MMAction2 to ONNX.
+```shell
+python tools/pytorch2onnx.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [--shape ${SHAPE}] \
+ [--verify] [--show] [--output-file ${OUTPUT_FILE}] [--is-localizer] [--opset-version ${VERSION}]
+```
+Optional arguments:
++ `--shape`: The shape of input tensor to the model. For 2D recognizer(e.g. TSN), the input should be `$batch $clip $channel $height $width`(e.g. `1 1 3 224 224`); For 3D recognizer(e.g. I3D), the input should be `$batch $clip $channel $time $height $width`(e.g. `1 1 3 32 224 224`); For localizer such as BSN, the input for each module is different, please check the `forward` function for it. If not specified, it will be set to `1 1 3 224 224`.
++ `--verify`: Determines whether to verify the exported model, runnably and numerically. If not specified, it will be set to `False`.
++ `--show`: Determines whether to print the architecture of the exported model. If not specified, it will be set to `False`.
++ `--output-file`: The output onnx model name. If not specified, it will be set to `tmp.onnx`.
++ `--is-localizer`: Determines whether the model to be exported is a localizer. If not specified, it will be set to `False`.
++ `--opset-version`: Determines the operation set version of onnx, we recommend you to use a higher version such as 11 for compatibility. If not specified, it will be set to `11`.
+
+### Recognizers
+For recognizers, please run:
+```shell
+python tools/pytorch2onnx.py $CONFIG_PATH $CHECKPOINT_PATH --shape $SHAPE --verify
+```
+
+### Localizer
+For localizers, please run:
+```shell
+python tools/pytorch2onnx.py $CONFIG_PATH $CHECKPOINT_PATH --is-localizer --shape $SHAPE --verify
+```
+
+Please fire an issue if you discover any checkpoints that are not perfectly exported or suffer some loss in accuracy.
diff --git a/Downstream/Open-Set-Action-Recognition/docs/tutorials/finetune.md b/Downstream/Open-Set-Action-Recognition/docs/tutorials/finetune.md
new file mode 100644
index 0000000..6b87c20
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/tutorials/finetune.md
@@ -0,0 +1,95 @@
+# Tutorial 1: Finetuning Models
+
+This tutorial provides instructions for users to use the pre-trained models
+to finetune them on other datasets, so that better performance can be get.
+
+
+
+- [Outline](#outline)
+- [Modify Head](#modify-head)
+- [Modify Dataset](#modify-dataset)
+- [Modify Training Schedule](#modify-training-schedule)
+- [Use Pre-Trained Model](#use-pre-trained-model)
+
+
+
+## Outline
+
+There are two steps to finetune a model on a new dataset.
+
+1. Add support for the new dataset. See [Tutorial 2: Adding New Dataset](new_dataset.md).
+1. Modify the configs. This will be discussed in this tutorial.
+
+For example, if the user want to finetune models pre-trained on Kinetics-400 Dataset to another dataset, say UCF101,
+then four parts in the config (see [here](../config.md)) needs attention.
+
+## Modify Head
+
+The `num_classes` in the `cls_head` need to be changed to the class number of the new dataset.
+The weights of the pre-trained models are reused except for the final prediction layer.
+So it is safe to change the class number.
+In our case, UCF101 has 101 classes.
+So we change it from 400 (class number of Kinetics-400) to 101.
+
+```python
+model = dict(
+ type='Recognizer2D',
+ backbone=dict(
+ type='ResNet',
+ pretrained='torchvision://resnet50',
+ depth=50,
+ norm_eval=False),
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=101, # change from 400 to 101
+ in_channels=2048,
+ spatial_type='avg',
+ consensus=dict(type='AvgConsensus', dim=1),
+ dropout_ratio=0.4,
+ init_std=0.01))
+```
+
+Note that the `pretrained='torchvision://resnet50'` setting is used for initializing backbone.
+If you are training a new model from ImageNet-pretrained weights, this is for you.
+However, this setting is not related to our task at hand.
+What we need is `load_from`, which will be discussed later.
+
+## Modify Dataset
+
+MMAction2 supports UCF101, Kinetics-400, Moments in Time, Multi-Moments in Time, THUMOS14,
+Something-Something V1&V2, ActivityNet Dataset.
+The users may need to adapt one of the above dataset to fit for their special datasets.
+In our case, UCF101 is already supported by various dataset types, like `RawframeDataset`,
+so we change the config as follows.
+
+```python
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'data/ucf101/rawframes_train/'
+data_root_val = 'data/ucf101/rawframes_val/'
+ann_file_train = 'data/ucf101/ucf101_train_list.txt'
+ann_file_val = 'data/ucf101/ucf101_val_list.txt'
+ann_file_test = 'data/ucf101/ucf101_val_list.txt'
+```
+
+## Modify Training Schedule
+
+Finetuning usually requires smaller learning rate and less training epochs.
+
+```python
+# optimizer
+optimizer = dict(type='SGD', lr=0.005, momentum=0.9, weight_decay=0.0001) # change from 0.01 to 0.005
+optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))
+# learning policy
+lr_config = dict(policy='step', step=[40, 80])
+total_epochs = 50 # change from 100 to 50
+checkpoint_config = dict(interval=5)
+```
+
+## Use Pre-Trained Model
+To use the pre-trained model for the whole network, the new config adds the link of pre-trained models in the `load_from`.
+
+```python
+# use the pre-trained model for the whole TSN network
+load_from = 'https://open-mmlab.s3.ap-northeast-2.amazonaws.com/mmaction/mmaction-v1/recognition/tsn_r50_1x1x3_100e_kinetics400_rgb/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth' # model path can be found in model zoo
+```
diff --git a/Downstream/Open-Set-Action-Recognition/docs/tutorials/new_dataset.md b/Downstream/Open-Set-Action-Recognition/docs/tutorials/new_dataset.md
new file mode 100644
index 0000000..e55844a
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/tutorials/new_dataset.md
@@ -0,0 +1,248 @@
+# Tutorial 2: Adding New Dataset
+
+In this tutorial, we will introduce some methods about how to customize your own dataset by reorganizing data and mixing dataset for the project.
+
+
+
+- [Customize Datasets by Reorganizing Data](#customize-datasets-by-reorganizing-data)
+ * [Reorganize datasets to existing format](#reorganize-datasets-to-existing-format)
+ * [An example of a custom dataset](#an-example-of-a-custom-dataset)
+- [Customize Dataset by Mixing Dataset](#customize-dataset-by-mixing-dataset)
+ * [Repeat dataset](#repeat-dataset)
+
+
+
+## Customize Datasets by Reorganizing Data
+
+### Reorganize datasets to existing format
+
+The simplest way is to convert your dataset to existing dataset formats (RawframeDataset or VideoDataset).
+
+There are three kinds of annotation files.
+
+- rawframe annotation
+
+ The annotation of a rawframe dataset is a text file with multiple lines,
+ and each line indicates `frame_directory` (relative path) of a video,
+ `total_frames` of a video and the `label` of a video, which are split by a whitespace.
+
+ Here is an example.
+ ```
+ some/directory-1 163 1
+ some/directory-2 122 1
+ some/directory-3 258 2
+ some/directory-4 234 2
+ some/directory-5 295 3
+ some/directory-6 121 3
+ ```
+
+- video annotation
+
+ The annotation of a video dataset is a text file with multiple lines,
+ and each line indicates a sample video with the `filepath` (relative path) and `label`,
+ which are split by a whitespace.
+
+ Here is an example.
+ ```
+ some/path/000.mp4 1
+ some/path/001.mp4 1
+ some/path/002.mp4 2
+ some/path/003.mp4 2
+ some/path/004.mp4 3
+ some/path/005.mp4 3
+ ```
+
+- ActivityNet annotation
+ The annotation of ActivityNet dataset is a json file. Each key is a video name
+ and the corresponding value is the meta data and annotation for the video.
+
+ Here is an example.
+ ```
+ {
+ "video1": {
+ "duration_second": 211.53,
+ "duration_frame": 6337,
+ "annotations": [
+ {
+ "segment": [
+ 30.025882995319815,
+ 205.2318595943838
+ ],
+ "label": "Rock climbing"
+ }
+ ],
+ "feature_frame": 6336,
+ "fps": 30.0,
+ "rfps": 29.9579255898
+ },
+ "video2": {
+ "duration_second": 26.75,
+ "duration_frame": 647,
+ "annotations": [
+ {
+ "segment": [
+ 2.578755070202808,
+ 24.914101404056165
+ ],
+ "label": "Drinking beer"
+ }
+ ],
+ "feature_frame": 624,
+ "fps": 24.0,
+ "rfps": 24.1869158879
+ }
+ }
+ ```
+
+There are two ways to work with custom datasets.
+
+- online conversion
+
+ You can write a new Dataset class inherited from [BaseDataset](/mmaction/datasets/base.py), and overwrite three methods
+ `load_annotations(self)`, `evaluate(self, results, metrics, logger)` and `dump_results(self, results, out)`,
+ like [RawframeDataset](/mmaction/datasets/rawframe_dataset.py), [VideoDataset](/mmaction/datasets/video_dataset.py) or [ActivityNetDataset](/mmaction/datasets/activitynet_dataset.py).
+
+- offline conversion
+
+ You can convert the annotation format to the expected format above and save it to
+ a pickle or json file, then you can simply use `RawframeDataset`, `VideoDataset` or `ActivityNetDataset`.
+
+After the data pre-processing, the users need to further modify the config files to use the dataset.
+Here is an example of using a custom dataset in rawframe format.
+
+In `configs/task/method/my_custom_config.py`:
+
+```python
+...
+# dataset settings
+dataset_type = 'RawframeDataset'
+data_root = 'path/to/your/root'
+data_root_val = 'path/to/your/root_val'
+ann_file_train = 'data/custom/custom_train_list.txt'
+ann_file_val = 'data/custom/custom_val_list.txt'
+ann_file_test = 'data/custom/custom_val_list.txt'
+...
+data = dict(
+ videos_per_gpu=32,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=ann_file_train,
+ ...),
+ val=dict(
+ type=dataset_type,
+ ann_file=ann_file_val,
+ ...),
+ test=dict(
+ type=dataset_type,
+ ann_file=ann_file_test,
+ ...))
+...
+```
+
+We use this way to support Rawframe dataset.
+
+### An example of a custom dataset
+
+Assume the annotation is in a new format in text files, and the image file name is of template like `img_00005.jpg`
+The video annotations are stored in text file `annotation.txt` as following
+
+```
+directory,total frames,class
+D32_1gwq35E,299,66
+-G-5CJ0JkKY,249,254
+T4h1bvOd9DA,299,33
+4uZ27ivBl00,299,341
+0LfESFkfBSw,249,186
+-YIsNpBEx6c,299,169
+```
+
+We can create a new dataset in `mmaction/datasets/my_dataset.py` to load the data.
+
+```python
+import copy
+import os.path as osp
+
+import mmcv
+
+from .base import BaseDataset
+from .registry import DATASETS
+
+
+@DATASETS.register_module()
+class MyDataset(BaseDataset):
+
+ def __init__(self,
+ ann_file,
+ pipeline,
+ data_prefix=None,
+ test_mode=False,
+ filename_tmpl='img_{:05}.jpg'):
+ super(MyDataset, self).__init__(ann_file, pipeline, test_mode)
+
+ self.filename_tmpl = filename_tmpl
+
+ def load_annotations(self):
+ video_infos = []
+ with open(self.ann_file, 'r') as fin:
+ for line in fin:
+ if line.startswith("directory"):
+ continue
+ frame_dir, total_frames, label = line.split(',')
+ if self.data_prefix is not None:
+ frame_dir = osp.join(self.data_prefix, frame_dir)
+ video_infos.append(
+ dict(
+ frame_dir=frame_dir,
+ total_frames=int(total_frames),
+ label=int(label)))
+ return video_infos
+
+ def prepare_train_frames(self, idx):
+ results = copy.deepcopy(self.video_infos[idx])
+ results['filename_tmpl'] = self.filename_tmpl
+ return self.pipeline(results)
+
+ def prepare_test_frames(self, idx):
+ results = copy.deepcopy(self.video_infos[idx])
+ results['filename_tmpl'] = self.filename_tmpl
+ return self.pipeline(results)
+
+ def evaluate(self,
+ results,
+ metrics='top_k_accuracy',
+ topk=(1, 5),
+ logger=None):
+ pass
+```
+
+Then in the config, to use `MyDataset` you can modify the config as the following
+
+```python
+dataset_A_train = dict(
+ type='MyDataset',
+ ann_file = ann_file_train,
+ pipeline=train_pipeline
+)
+```
+
+## Customize Dataset by Mixing Dataset
+
+MMAction2 also supports to mix dataset for training. Currently it supports to repeat dataset.
+
+### Repeat dataset
+
+We use `RepeatDataset` as wrapper to repeat the dataset. For example, suppose the original dataset as `Dataset_A`,
+to repeat it, the config looks like the following
+
+```python
+dataset_A_train = dict(
+ type='RepeatDataset',
+ times=N,
+ dataset=dict( # This is the original config of Dataset_A
+ type='Dataset_A',
+ ...
+ pipeline=train_pipeline
+ )
+ )
+```
diff --git a/Downstream/Open-Set-Action-Recognition/docs/tutorials/new_modules.md b/Downstream/Open-Set-Action-Recognition/docs/tutorials/new_modules.md
new file mode 100644
index 0000000..32be1b6
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/tutorials/new_modules.md
@@ -0,0 +1,283 @@
+# Tutorial 4: Adding New Modules
+
+In this tutorial, we will introduce some methods about how to customize optimizer, develop new components and new a learning rate scheduler for this project.
+
+
+
+- [Customize Optimizer](#customize-optimizer)
+- [Customize Optimizer Constructor](#customize-optimizer-constructor)
+- [Develop New Components](#develop-new-components)
+ * [Add new backbones](#add-new-backbones)
+ * [Add new heads](#add-new-heads)
+ * [Add new loss](#add-new-loss)
+- [Add new learning rate scheduler (updater)](#add-new-learning-rate-scheduler--updater-)
+
+
+
+## Customize Optimizer
+
+An example of customized optimizer is [CopyOfSGD](/mmaction/core/optimizer/copy_of_sgd.py) is defined in `mmaction/core/optimizer/copy_of_sgd.py`.
+More generally, a customized optimizer could be defined as following.
+
+Assume you want to add an optimizer named as `MyOptimizer`, which has arguments `a`, `b` and `c`.
+You need to first implement the new optimizer in a file, e.g., in `mmaction/core/optimizer/my_optimizer.py`:
+
+```python
+from .registry import OPTIMIZERS
+from torch.optim import Optimizer
+
+@OPTIMIZERS.register_module()
+class MyOptimizer(Optimizer):
+
+ def __init__(self, a, b, c):
+```
+
+Then add this module in `mmaction/core/optimizer/__init__.py`, thus the registry will find the new module and add it:
+
+```python
+from .my_optimizer import MyOptimizer
+```
+
+Then you can use `MyOptimizer` in `optimizer` field of config files.
+In the configs, the optimizers are defined by the field `optimizer` like the following:
+
+```python
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+```
+
+To use your own optimizer, the field can be changed as
+
+```python
+optimizer = dict(type='MyOptimizer', a=a_value, b=b_value, c=c_value)
+```
+
+We already support to use all the optimizers implemented by PyTorch, and the only modification is to change the `optimizer` field of config files.
+For example, if you want to use `ADAM`, though the performance will drop a lot, the modification could be as the following.
+
+```python
+optimizer = dict(type='Adam', lr=0.0003, weight_decay=0.0001)
+```
+
+The users can directly set arguments following the [API doc](https://pytorch.org/docs/stable/optim.html?highlight=optim#module-torch.optim) of PyTorch.
+
+## Customize Optimizer Constructor
+
+Some models may have some parameter-specific settings for optimization, e.g. weight decay for BatchNorm layers.
+The users can do those fine-grained parameter tuning through customizing optimizer constructor.
+
+You can write a new optimizer constructor inherit from [DefaultOptimizerConstructor](https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/optimizer/default_constructor.py)
+and overwrite the `add_params(self, params, module)` method.
+
+An example of customized optimizer constructor is [TSMOptimizerConstructor](/mmaction/core/optimizer/tsm_optimizer_constructor.py).
+More generally, a customized optimizer constructor could be defined as following.
+
+In `mmaction/core/optimizer/my_optimizer_constructor.py`:
+
+```python
+from mmcv.runner import OPTIMIZER_BUILDERS, DefaultOptimizerConstructor
+
+@OPTIMIZER_BUILDERS.register_module()
+class MyOptimizerConstructor(DefaultOptimizerConstructor):
+
+```
+
+In `mmaction/core/optimizer/__init__.py`:
+
+```python
+from .my_optimizer_constructor import MyOptimizerConstructor
+```
+
+Then you can use `MyOptimizerConstructor` in `optimizer` field of config files.
+
+```python
+# optimizer
+optimizer = dict(
+ type='SGD',
+ constructor='MyOptimizerConstructor',
+ paramwise_cfg=dict(fc_lr5=True),
+ lr=0.02,
+ momentum=0.9,
+ weight_decay=0.0001)
+```
+
+## Develop New Components
+
+We basically categorize model components into 4 types.
+
+- recognizer: the whole recognizer model pipeline, usually contains a backbone and cls_head.
+- backbone: usually an FCN network to extract feature maps, e.g., ResNet, BNInception.
+- cls_head: the component for classification task, usually contains an FC layer with some pooling layers.
+- localizer: the model for temporal localization task, currently available: BSN, BMN.
+
+### Add new backbones
+
+Here we show how to develop new components with an example of TSN.
+
+1. Create a new file `mmaction/models/backbones/resnet.py`.
+
+ ```python
+ import torch.nn as nn
+
+ from ..registry import BACKBONES
+
+ @BACKBONES.register_module()
+ class ResNet(nn.Module):
+
+ def __init__(self, arg1, arg2):
+ pass
+
+ def forward(self, x): # should return a tuple
+ pass
+
+ def init_weights(self, pretrained=None):
+ pass
+ ```
+
+2. Import the module in `mmaction/models/backbones/__init__.py`.
+
+ ```python
+ from .resnet import ResNet
+ ```
+
+3. Use it in your config file.
+
+ ```python
+ model = dict(
+ ...
+ backbone=dict(
+ type='ResNet',
+ arg1=xxx,
+ arg2=xxx),
+ )
+ ```
+
+### Add new heads
+
+Here we show how to develop a new head with the example of TSNHead as the following.
+
+1. Create a new file `mmaction/models/heads/tsn_head.py`.
+
+ You can write a new classification head inheriting from [BaseHead](/mmaction/models/heads/base.py),
+ and overwrite `init_weights(self)` and `forward(self, x)` method.
+
+ ```python
+ from ..registry import HEADS
+ from .base import BaseHead
+
+
+ @HEADS.register_module()
+ class TSNHead(BaseHead):
+
+ def __init__(self, arg1, arg2):
+ pass
+
+ def forward(self, x):
+ pass
+
+ def init_weights(self):
+ pass
+ ```
+
+2. Import the module in `mmaction/models/heads/__init__.py`
+
+ ```python
+ from .tsn_head import TSNHead
+ ```
+
+3. Use it in your config file
+
+ ```python
+ model = dict(
+ ...
+ cls_head=dict(
+ type='TSNHead',
+ num_classes=400,
+ in_channels=2048,
+ arg1=xxx,
+ arg2=xxx),
+ ```
+
+### Add new loss
+
+Assume you want to add a new loss as `MyLoss`. To add a new loss function, the users need implement it in `mmaction/models/losses/my_loss.py`.
+
+```python
+import torch
+import torch.nn as nn
+
+from ..builder import LOSSES
+
+def my_loss(pred, target):
+ assert pred.size() == target.size() and target.numel() > 0
+ loss = torch.abs(pred - target)
+ return loss
+
+
+@LOSSES.register_module()
+class MyLoss(nn.Module):
+
+ def forward(self, pred, target):
+ loss = my_loss(pred, target)
+ return loss
+```
+
+Then the users need to add it in the `mmaction/models/losses/__init__.py`
+
+```python
+from .my_loss import MyLoss, my_loss
+```
+
+To use it, modify the `loss_xxx` field. Since MyLoss is for regression, we can use it for the bbox loss `loss_bbox`.
+
+```python
+loss_bbox=dict(type='MyLoss'))
+```
+
+## Add new learning rate scheduler (updater)
+The default manner of constructing a lr updater(namely, 'scheduler' by pytorch convention), is to modify the config such as:
+```python
+...
+lr_config = dict(policy='step', step=[20, 40])
+...
+```
+In the api for [`train.py`](/mmaction/apis/train.py), it will register the learning rate updater hook based on the config at:
+```python
+...
+ runner.register_training_hooks(
+ cfg.lr_config,
+ optimizer_config,
+ cfg.checkpoint_config,
+ cfg.log_config,
+ cfg.get('momentum_config', None))
+...
+```
+So far, the supported updaters can be find in [mmcv](https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/lr_updater.py), but if you want to customize a new learning rate updater, you may follow the steps below:
+
+1. First, write your own LrUpdaterHook in `$MMAction2/mmaction/core/lr`. The snippet followed is an example of cumtomized lr updater that uses learning rate based on a specific learning rate ratio: `lrs`, by which the learning rate decreases at each `steps`:
+```python
+@HOOKS.register_module()
+# Register it here
+class RelativeStepLrUpdaterHook(LrUpdaterHook):
+ # You should inheritate it from mmcv.LrUpdaterHook
+ def __init__(self, runner, steps, lrs, **kwargs):
+ super().__init__(**kwargs)
+ assert len(steps) == (len(lrs))
+ self.steps = steps
+ self.lrs = lrs
+
+ def get_lr(self, runner, base_lr):
+ # Only this function is required to override
+ # This function is called before each training epoch, return the specific learning rate here.
+ progress = runner.epoch if self.by_epoch else runner.iter
+ for i in range(len(self.steps)):
+ if progress < self.steps[i]:
+ return self.lrs[i]
+```
+
+2. Modify your config:
+In your config file, swap the original `lr_config` by:
+```python
+lr_config = dict(policy='RelativeStep', steps=[20, 40, 60], lrs=[0.1, 0.01, 0.001])
+```
+
+More examples can be found in [mmcv](https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/lr_updater.py).
diff --git a/Downstream/Open-Set-Action-Recognition/docs/useful_tools.md b/Downstream/Open-Set-Action-Recognition/docs/useful_tools.md
new file mode 100644
index 0000000..7ad6c8d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/docs/useful_tools.md
@@ -0,0 +1,155 @@
+Apart from training/testing scripts, We provide lots of useful tools under the `tools/` directory.
+
+## Useful Tools Link
+
+
+
+- [Log Analysis](#log-analysis)
+- [Model Complexity](#model-complexity)
+- [Model Conversion](#model-conversion)
+ * [MMAction2 model to ONNX (experimental)](#mmaction2-model-to-onnx--experimental-)
+ * [Prepare a model for publishing](#prepare-a-model-for-publishing)
+- [Miscellaneous](#miscellaneous)
+ * [Evaluating a metric](#evaluating-a-metric)
+ * [Print the entire config](#print-the-entire-config)
+
+
+
+## Log Analysis
+
+`tools/analysis/analyze_logs.py` plots loss/top-k acc curves given a training log file. Run `pip install seaborn` first to install the dependency.
+
+![acc_curve_image](imgs/acc_curve.png)
+
+```shell
+python tools/analysis/analyze_logs.py plot_curve ${JSON_LOGS} [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}]
+```
+
+Examples:
+
+- Plot the classification loss of some run.
+
+ ```shell
+ python tools/analysis/analyze_logs.py plot_curve log.json --keys loss_cls --legend loss_cls
+ ```
+
+- Plot the top-1 acc and top-5 acc of some run, and save the figure to a pdf.
+
+ ```shell
+ python tools/analysis/analyze_logs.py plot_curve log.json --keys top1_acc top5_acc --out results.pdf
+ ```
+
+- Compare the top-1 acc of two runs in the same figure.
+
+ ```shell
+ python tools/analysis/analyze_logs.py plot_curve log1.json log2.json --keys top1_acc --legend run1 run2
+ ```
+
+You can also compute the average training speed.
+
+ ```shell
+ python tools/analysis/analyze_logs.py cal_train_time ${JSON_LOGS} [--include-outliers]
+ ```
+
+- Compute the average training speed for a config file
+
+ ```shell
+ python tools/analysis/analyze_logs.py cal_train_time work_dirs/some_exp/20200422_153324.log.json
+ ```
+
+ The output is expected to be like the following.
+
+ ```text
+ -----Analyze train time of work_dirs/some_exp/20200422_153324.log.json-----
+ slowest epoch 60, average time is 0.9736
+ fastest epoch 18, average time is 0.9001
+ time std over epochs is 0.0177
+ average iter time: 0.9330 s/iter
+ ```
+
+## Model Complexity
+
+`/tools/analysis/get_flops.py` is a script adapted from [flops-counter.pytorch](https://github.com/sovrasov/flops-counter.pytorch) to compute the FLOPs and params of a given model.
+
+```shell
+python tools/analysis/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}]
+```
+
+We will get the result like this
+
+```text
+==============================
+Input shape: (1, 3, 32, 340, 256)
+Flops: 37.1 GMac
+Params: 28.04 M
+==============================
+```
+
+**Note**: This tool is still experimental and we do not guarantee that the number is absolutely correct.
+You may use the result for simple comparisons, but double check it before you adopt it in technical reports or papers.
+
+(1) FLOPs are related to the input shape while parameters are not. The default input shape is (1, 3, 340, 256) for 2D recognizer, (1, 3, 32, 340, 256) for 3D recognizer.
+(2) Some operators are not counted into FLOPs like GN and custom operators. Refer to [`mmcv.cnn.get_model_complexity_info()`](https://github.com/open-mmlab/mmcv/blob/master/mmcv/cnn/utils/flops_counter.py) for details.
+
+## Model Conversion
+
+### MMAction2 model to ONNX (experimental)
+
+`/tools/pytorch2onnx.py` is a script to convert model to [ONNX](https://github.com/onnx/onnx) format.
+It also supports comparing the output results between Pytorch and ONNX model for verification.
+Run `pip install onnx onnxruntime` first to install the dependency.
+
+- For recognizers, please run:
+
+ ```shell
+ python tools/pytorch2onnx.py $CONFIG_PATH $CHECKPOINT_PATH --shape $SHAPE --verify
+ ```
+
+- For localizers, please run:
+
+ ```shell
+ python tools/pytorch2onnx.py $CONFIG_PATH $CHECKPOINT_PATH --is-localizer --shape $SHAPE --verify
+ ```
+
+### Prepare a model for publishing
+
+`tools/publish_model.py` helps users to prepare their model for publishing.
+
+Before you upload a model to AWS, you may want to:
+
+(1) convert model weights to CPU tensors.
+(2) delete the optimizer states.
+(3) compute the hash of the checkpoint file and append the hash id to the filename.
+
+```shell
+python tools/publish_model.py ${INPUT_FILENAME} ${OUTPUT_FILENAME}
+```
+
+E.g.,
+
+```shell
+python tools/publish_model.py work_dirs/tsn_r50_1x1x3_100e_kinetics400_rgb/latest.pth tsn_r50_1x1x3_100e_kinetics400_rgb.pth
+```
+
+The final output filename will be `tsn_r50_1x1x3_100e_kinetics400_rgb-{hash id}.pth`.
+
+## Miscellaneous
+
+### Evaluating a metric
+
+tools/analysis/eval_metric.py evaluates certain metrics of the results saved in a file according to a config file.
+
+The saved result file is created on `tools/test.py` by setting the arguments `--out ${RESULT_FILE}` to indicate the result file,
+which stores the final output of the whole model.
+
+```shell
+python tools/analysis/eval_metric.py ${CONFIG_FILE} ${RESULT_FILE} [--eval ${EVAL_METRICS}] [--cfg-options ${CFG_OPTIONS}] [--eval-options ${EVAL_OPTIONS}]
+```
+
+### Print the entire config
+
+`tools/analysis/print_config.py` prints the whole config verbatim, expanding all its imports.
+
+```shell
+python tools/print_config.py ${CONFIG} [-h] [--options ${OPTIONS [OPTIONS...]}]
+```
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/analyze_features.py b/Downstream/Open-Set-Action-Recognition/experiments/analyze_features.py
new file mode 100644
index 0000000..1ab4110
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/analyze_features.py
@@ -0,0 +1,163 @@
+import argparse
+import os
+import torch
+from mmcv.parallel import collate, scatter
+from mmaction.datasets.pipelines import Compose
+from mmaction.apis import init_recognizer
+from sklearn.manifold import TSNE
+import numpy as np
+import matplotlib.pyplot as plt
+from tqdm import tqdm
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='MMAction2 test')
+ # model config
+ parser.add_argument('--config', help='test config file path')
+ parser.add_argument('--checkpoint', help='checkpoint file/url')
+ parser.add_argument('--known_split', help='the split file path of the knowns')
+ parser.add_argument('--unknown_split', help='the split file path of the unknowns')
+ parser.add_argument('--result_file', help='the result file path')
+ parser.add_argument('--device', type=str, default='cuda:0', help='CPU/CUDA device option')
+ args = parser.parse_args()
+ return args
+
+
+def get_data(known_split, known_classes):
+ known_data = []
+ labels = []
+ video_dir = os.path.join(os.path.dirname(known_split), 'videos')
+ with open(known_split, 'r') as f:
+ for line in f.readlines():
+ clsname, videoname = line.strip().split(' ')[0].split('/')
+ if clsname in known_classes.keys():
+ videofile = os.path.join(video_dir, clsname, videoname)
+ known_data.append(videofile)
+ labels.append(known_classes[clsname])
+ return known_data, labels
+
+
+def inference_recognizer(model, video_path):
+ """Inference a video with the detector.
+
+ Args:
+ model (nn.Module): The loaded recognizer.
+ video_path (str): The video file path/url or the rawframes directory
+ path. If ``use_frames`` is set to True, it should be rawframes
+ directory path. Otherwise, it should be video file path.
+ """
+ cfg = model.cfg
+ device = next(model.parameters()).device # model device
+ # build the data pipeline
+ test_pipeline = cfg.data.test.pipeline
+ test_pipeline = Compose(test_pipeline)
+ # prepare data (by default, we use videodata)
+ start_index = cfg.data.test.get('start_index', 0)
+ data = dict(filename=video_path, label=-1, start_index=start_index, modality='RGB')
+ data = test_pipeline(data)
+ data = collate([data], samples_per_gpu=1)
+ if next(model.parameters()).is_cuda:
+ # scatter to specified GPU
+ data = scatter(data, [device])[0]
+
+ # forward the model
+ with torch.no_grad():
+ feat_blob = model(return_loss=False, get_feat=True, **data) # (num_clips * num_crops, 2048, 1, 8, 8)
+ # spatial average pooling
+ kernel_size = (1, feat_blob.size(-2), feat_blob.size(-1))
+ avg_pool2d = torch.nn.AvgPool3d(kernel_size, stride=1, padding=0)
+ feat_clips = avg_pool2d(feat_blob).view(feat_blob.size(0), feat_blob.size(1)) # (num_clips * num_crops, 2048)
+ # get the mean features of all clips and crops
+ feat_final = torch.mean(feat_clips, dim=0).cpu().numpy() # (2048,)
+ return feat_final
+
+def extract_feature(video_files):
+
+ model = init_recognizer(
+ args.config,
+ args.checkpoint,
+ device=device,
+ use_frames=False)
+ cfg = model.cfg
+ torch.backends.cudnn.benchmark = True
+ cfg.data.test.test_mode = True
+ if 'bnn' in args.config:
+ model.test_cfg.npass = 1
+
+ X = []
+ for videofile in tqdm(video_files, total=len(video_files), desc='Extract Feature'):
+ feature = inference_recognizer(model, videofile) # (2048,)
+ X.append(feature)
+ X = np.vstack(X)
+ return X
+
+
+def set_deterministic(seed):
+ torch.manual_seed(seed)
+ torch.cuda.manual_seed(seed)
+ torch.cuda.manual_seed_all(seed) # if you are using multi-GPU.
+ np.random.seed(seed) # Numpy module.
+ torch.backends.cudnn.benchmark = True
+ torch.backends.cudnn.deterministic = True
+
+
+if __name__ == '__main__':
+
+ args = parse_args()
+ # assign the desired device.
+ device = torch.device(args.device)
+ set_deterministic(0)
+
+ ind_clsID = [2, 10, 16, 69, 71, 21, 32, 41, 73, 29] # UCF-101 73 21 41 32 29 10 16 69 71 2
+ ind_classes = {'Archery': 0, 'Biking': 1, 'BoxingPunchingBag': 2, 'PullUps': 3, 'PushUps': 4,
+ 'CliffDiving': 5, 'GolfSwing': 6, 'HorseRiding': 7, 'RockClimbingIndoor': 8, 'FloorGymnastics': 9}
+ ood_clsID = [12, 20, 21, 22, 50, 15, 16, 17, 18, 19] # HMDB-51 15 16 17 18 19 20 21 22 12 50
+ ood_classes = {'fall_floor': 10, 'kick': 10, 'kick_ball': 10, 'kiss': 10, 'wave': 10,
+ 'golf': 10, 'handstand': 10, 'hit': 10, 'hug': 10, 'jump': 10}
+ feature_file = args.result_file[:-4] + '_feature_10p1.npz'
+
+ # get the data of known classes
+ known_data, known_labels = get_data(args.known_split, ind_classes)
+ num_knowns = len(known_data)
+ # get the data of unknown classes
+ unknown_data, unknown_labels = get_data(args.unknown_split, ood_classes)
+ num_unknowns = len(unknown_data)
+
+ if not os.path.exists(feature_file):
+ # save the figure
+ result_path = os.path.dirname(args.result_file)
+ if not os.path.exists(result_path):
+ os.makedirs(result_path)
+
+ # extracting the feature
+ X = extract_feature(known_data + unknown_data)
+
+ # save results
+ np.savez(feature_file[:-4], feature=X)
+ else:
+ results = np.load(feature_file, allow_pickle=True)
+ X = results['feature']
+
+ open_classes = {**ind_classes, 'Unknowns': len(ind_classes)}
+ open_labels = np.array(known_labels + [len(ind_classes)] * num_unknowns)
+ # run tSNE
+ print('running tSNE...')
+ Y = TSNE(n_components=2, random_state=0).fit_transform(X)
+ plt.figure(figsize=(5,4))
+ plt.rcParams["font.family"] = "Arial" # Times New Roman
+ fontsize = 10
+ for k, v in open_classes.items():
+ inds = np.where(open_labels == v)[0]
+ if k == 'Unknowns':
+ plt.scatter(Y[inds, 0], Y[inds, 1], s=10, c='k', marker='^', label=k)
+ else:
+ plt.scatter(Y[inds, 0], Y[inds, 1], s=3)
+ plt.text(np.mean(Y[inds, 0])-5, np.mean(Y[inds, 1])+5, k, fontsize=fontsize)
+ xmin, xmax, ymin, ymax = np.min(Y[:, 0]), np.max(Y[:, 0]), np.min(Y[:, 1]), np.max(Y[:, 1])
+ plt.xlim(xmin-5, xmax + 15)
+ plt.ylim(ymin-5, ymax + 10)
+ plt.legend(loc='lower right', fontsize=fontsize)
+ plt.xticks([])
+ plt.yticks([])
+ plt.savefig(args.result_file)
+ plt.savefig(args.result_file[:-4] + '.pdf')
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/baseline_openmax.py b/Downstream/Open-Set-Action-Recognition/experiments/baseline_openmax.py
new file mode 100644
index 0000000..394b318
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/baseline_openmax.py
@@ -0,0 +1,436 @@
+import argparse, os, sys
+import torch
+import mmcv
+import numpy as np
+import torch.nn.functional as F
+from mmcv.parallel import collate, scatter
+from mmaction.datasets.pipelines import Compose
+from mmaction.apis import init_recognizer
+from mmaction.datasets import build_dataloader, build_dataset
+from mmcv.parallel import MMDataParallel
+from tqdm import tqdm
+import scipy.spatial.distance as spd
+try:
+ import libmr
+except ImportError:
+ print("LibMR not installed or libmr.so not found")
+ print("Install libmr: cd libMR/; ./compile.sh")
+ sys.exit()
+from sklearn.metrics import f1_score, roc_auc_score, accuracy_score
+import matplotlib.pyplot as plt
+
+
+def set_deterministic(seed):
+ torch.manual_seed(seed)
+ torch.cuda.manual_seed(seed)
+ torch.cuda.manual_seed_all(seed) # if you are using multi-GPU.
+ np.random.seed(seed) # Numpy module.
+ torch.backends.cudnn.benchmark = True
+ torch.backends.cudnn.deterministic = True
+
+def parse_args():
+ """ Example shell script:
+ $ cd experiments
+ $ source activate mmaction
+ $ nohup python baseline_openmax.py --model i3d --config configs/recognition/tpn/inference_tpn_slowonly_dnn.py
+ """
+ parser = argparse.ArgumentParser(description='MMAction2 test')
+ # model config
+ parser.add_argument('--config', help='test config file path')
+ parser.add_argument('--checkpoint', help='checkpoint file/url')
+ parser.add_argument('--trainset_split', default='data/ucf101/ucf101_train_split_1_videos.txt', help='the split file path of the training set')
+ parser.add_argument('--num_cls', type=int, default=101, help='The number of classes in training set.')
+ parser.add_argument('--cache_mav_dist', help='the result path to cache the mav and distances for each class.')
+ # test data config
+ parser.add_argument('--ind_data', help='the split file of in-distribution testing data')
+ parser.add_argument('--ood_data', help='the split file of out-of-distribution testing data')
+ parser.add_argument('--ood_ncls', type=int, help='the number of classes in unknwon dataset')
+ parser.add_argument('--num_rand', type=int, default=10, help='the number of random selection for ood classes')
+ # device
+ parser.add_argument('--device', type=str, default='cuda:0', help='CPU/CUDA device option')
+ parser.add_argument('--result_prefix', help='result file prefix')
+ args = parser.parse_args()
+ return args
+
+
+def get_datalist(split_file):
+ assert os.path.exists(split_file), 'split file does not exist! %s'%(split_file)
+ video_dir = os.path.join(os.path.dirname(split_file), 'videos')
+ filelist, labels = [], []
+ with open(split_file, 'r') as f:
+ for line in f.readlines():
+ videofile = os.path.join(video_dir, line.strip().split(' ')[0])
+ clsid = int(line.strip().split(' ')[-1])
+ filelist.append(videofile)
+ labels.append(clsid)
+ return filelist, labels
+
+def spatial_temporal_pooling(feat_blob):
+ if isinstance(feat_blob, tuple): # slowfast model returns a tuple of features
+ assert len(feat_blob) == 2, "invalid feature tuple!"
+ avg_pool3d = torch.nn.AdaptiveAvgPool3d((1, 1, 1))
+ x_fast, x_slow = feat_blob
+ x_fast = avg_pool3d(x_fast)
+ x_slow = avg_pool3d(x_slow)
+ # [N, channel_fast + channel_slow, 1, 1, 1]
+ feat_clips = torch.cat((x_slow, x_fast), dim=1).squeeze(-1).squeeze(-1).squeeze(-1)
+ else:
+ if len(feat_blob.size()) == 5: # 3D Network
+ # spatial temporal average pooling
+ kernel_size = (feat_blob.size(-3), feat_blob.size(-2), feat_blob.size(-1))
+ avg_pool3d = torch.nn.AvgPool3d(kernel_size, stride=1, padding=0)
+ feat_clips = avg_pool3d(feat_blob).view(feat_blob.size(0), feat_blob.size(1)) # (c, D)
+ elif len(feat_blob.size()) == 4: # 2D Network
+ # spatial temporal average pooling
+ kernel_size = (feat_blob.size(-2), feat_blob.size(-1))
+ avg_pool2d = torch.nn.AvgPool2d(kernel_size, stride=1, padding=0)
+ feat_clips = avg_pool2d(feat_blob).view(feat_blob.size(0), feat_blob.size(1)) # (c, D)
+ else:
+ print('Unsupported feature dimension: {}'.format(feat_blob.size()))
+ # get the mean features of all clips and crops
+ feat_final = torch.mean(feat_clips, dim=0, keepdim=True) # (c=1, D)
+ return feat_final
+
+def inference_recognizer(model, video_path):
+ """Inference a video with the detector.
+
+ Args:
+ model (nn.Module): The loaded recognizer.
+ video_path (str): The video file path/url or the rawframes directory
+ path. If ``use_frames`` is set to True, it should be rawframes
+ directory path. Otherwise, it should be video file path.
+ """
+ cfg = model.cfg
+ device = next(model.parameters()).device # model device
+ # build the data pipeline
+ test_pipeline = cfg.data.test.pipeline
+ test_pipeline = Compose(test_pipeline)
+ # prepare data (by default, we use videodata)
+ start_index = cfg.data.test.get('start_index', 0)
+ data = dict(filename=video_path, label=-1, start_index=start_index, modality='RGB')
+ data = test_pipeline(data)
+ data = collate([data], samples_per_gpu=1)
+ if next(model.parameters()).is_cuda:
+ # scatter to specified GPU
+ data = scatter(data, [device])[0]
+
+ # forward the model
+ with torch.no_grad():
+ feat_blob, score = model(return_loss=False, return_score=True, get_feat=True, **data) # (c, D, t, h, w)
+ feat_blob = spatial_temporal_pooling(feat_blob)
+ feat_final = feat_blob.cpu().numpy()
+ score = score.cpu().numpy()
+ return feat_final, score
+
+
+def extract_class_features(videolist, model, cls_gt):
+ features = []
+ for videofile in tqdm(videolist, total=len(videolist), desc='Extract Class %d Features'%(cls_gt)):
+ feat, score = inference_recognizer(model, videofile) # (c, D)
+ cls_pred = np.argmax(score, axis=1)
+ if cls_gt in cls_pred:
+ features.append(feat)
+ features = np.array(features) # (N, c, D)
+ return features
+
+
+def compute_distance(mav, features):
+ # extract features and compute distances for each class
+ num_channels = mav.shape[0]
+ eucos_dist, eu_dist, cos_dist = [], [], []
+ for feat in features:
+ # compute distance of each channel
+ eu_channel, cos_channel, eu_cos_channel = [], [], []
+ for c in range(num_channels):
+ eu_channel += [spd.euclidean(mav[c, :], feat[c, :])/200.]
+ cos_channel += [spd.cosine(mav[c, :], feat[c, :])]
+ eu_cos_channel += [spd.euclidean(mav[c, :], feat[c, :]) / 200.
+ + spd.cosine(mav[c, :], feat[c, :])] # Here, 200 is from the official OpenMax code
+ eu_dist += [eu_channel]
+ cos_dist += [cos_channel]
+ eucos_dist += [eu_cos_channel]
+ return np.array(eucos_dist), np.array(eu_dist), np.array(cos_dist)
+
+
+def compute_channel_distance(mav_channel, feat_channel, distance_type='eucos'):
+ if distance_type == 'eucos':
+ query_distance = spd.euclidean(mav_channel, feat_channel)/200. + spd.cosine(mav_channel, feat_channel)
+ elif distance_type == 'eu':
+ query_distance = spd.euclidean(mav_channel, feat_channel)/200.
+ elif distance_type == 'cos':
+ query_distance = spd.cosine(mav_channel, feat_channel)
+ else:
+ print("distance type not known: enter either of eucos, euclidean or cosine")
+ return query_distance
+
+
+def compute_mav_dist(videolist, labels, model, mav_dist_cachedir):
+ num_cls = model.cls_head.num_classes
+ mav_dist_list = []
+ for cls_gt in range(num_cls):
+ mav_dist_file = os.path.join(mav_dist_cachedir, 'mav_dist_cls%03d.npz'%(cls_gt))
+ mav_dist_list.append(mav_dist_file)
+ if os.path.exists(mav_dist_file):
+ continue
+ # data for the current class
+ inds = np.where(np.array(labels) == cls_gt)[0]
+ videos_cls = [videolist[i] for i in inds]
+ # extract MAV features
+ features = extract_class_features(videos_cls, model, cls_gt)
+ mav_train = np.mean(features, axis=0)
+ # compute distance
+ eucos_dist, eu_dist, cos_dist = compute_distance(mav_train, features)
+ # save MAV and distances
+ np.savez(mav_dist_file[:-4], mav=mav_train, eucos=eucos_dist, eu=eu_dist, cos=cos_dist)
+ return mav_dist_list
+
+
+def weibull_fitting(mav_dist_list, distance_type='eucos', tailsize=20):
+ weibull_model = {}
+ for cls_gt in range(len(mav_dist_list)):
+ # load the mav_dist file
+ cache = np.load(mav_dist_list[cls_gt], allow_pickle=True)
+ mav_train = cache['mav']
+ distances = cache[distance_type]
+
+ weibull_model[cls_gt] = {}
+ weibull_model[cls_gt]['mean_vec'] = mav_train
+
+ # weibull fitting for each channel
+ weibull_model[cls_gt]['weibull_model'] = []
+ num_channels = mav_train.shape[0]
+ for c in range(num_channels):
+ mr = libmr.MR()
+ tailtofit = sorted(distances[:, c])[-tailsize:]
+ mr.fit_high(tailtofit, len(tailtofit))
+ weibull_model[cls_gt]['weibull_model'] += [mr]
+ return weibull_model
+
+
+def compute_openmax_prob(openmax_score, openmax_score_u):
+ num_channels, num_cls = openmax_score.shape
+ prob_scores, prob_unknowns = [], []
+ for c in range(num_channels):
+ channel_scores, channel_unknowns = [], []
+ for gt_cls in range(num_cls):
+ channel_scores += [np.exp(openmax_score[c, gt_cls])]
+
+ total_denominator = np.sum(np.exp(openmax_score[c, :])) + np.exp(np.sum(openmax_score_u[c, :]))
+ prob_scores += [channel_scores/total_denominator ]
+ prob_unknowns += [np.exp(np.sum(openmax_score_u[c, :]))/total_denominator]
+
+ prob_scores = np.array(prob_scores)
+ prob_unknowns = np.array(prob_unknowns)
+
+ scores = np.mean(prob_scores, axis=0)
+ unknowns = np.mean(prob_unknowns, axis=0)
+ modified_scores = scores.tolist() + [unknowns]
+ assert len(modified_scores) == num_cls + 1
+ modified_scores = np.expand_dims(np.array(modified_scores), axis=0)
+ return modified_scores
+
+
+def openmax_recalibrate(weibull_model, feature, score, rank=1, distance_type='eucos'):
+ num_channels, num_cls = score.shape
+ # get the ranked alpha
+ alpharank = min(num_cls, rank)
+ ranked_list = np.mean(score, axis=0).argsort().ravel()[::-1]
+ alpha_weights = [((alpharank+1) - i)/float(alpharank) for i in range(1, alpharank+1)]
+ ranked_alpha = np.zeros((num_cls,))
+ for i in range(len(alpha_weights)):
+ ranked_alpha[ranked_list[i]] = alpha_weights[i]
+ # calibrate
+ openmax_score, openmax_score_u = [], []
+ for c in range(num_channels):
+ channel_scores = score[c, :]
+ openmax_channel = []
+ openmax_unknown = []
+ for cls_gt in range(num_cls):
+ # get distance between current channel and mean vector
+ mav_train = weibull_model[cls_gt]['mean_vec']
+ category_weibull = weibull_model[cls_gt]['weibull_model']
+ channel_distance = compute_channel_distance(mav_train[c, :], feature[c, :], distance_type=distance_type)
+ # obtain w_score for the distance and compute probability of the distance
+ wscore = category_weibull[c].w_score(channel_distance)
+ modified_score = channel_scores[cls_gt] * ( 1 - wscore*ranked_alpha[cls_gt] )
+ openmax_channel += [modified_score]
+ openmax_unknown += [channel_scores[cls_gt] - modified_score]
+ # gather modified scores for each channel
+ openmax_score += [openmax_channel]
+ openmax_score_u += [openmax_unknown]
+ openmax_score = np.array(openmax_score)
+ openmax_score_u = np.array(openmax_score_u)
+ # Pass the recalibrated scores into openmax
+ openmax_prob = compute_openmax_prob(openmax_score, openmax_score_u)
+
+ return openmax_prob
+
+
+def run_inference(model, weibull_model, datalist_file):
+ # switch config for different dataset
+ cfg = model.cfg
+ cfg.data.test.ann_file = datalist_file
+ cfg.data.test.data_prefix = os.path.join(os.path.dirname(datalist_file), 'videos')
+ cfg.test_cfg.average_clips = 'score' # we only need scores before softmax layer
+ model.cfg.data.videos_per_gpu = 1
+ model.cfg.data.workers_per_gpu = 0
+ num_cls = model.cls_head.num_classes
+
+ # build the dataloader
+ dataset = build_dataset(cfg.data.test, dict(test_mode=True))
+ dataloader_setting = dict(
+ videos_per_gpu=cfg.data.get('videos_per_gpu', 1),
+ workers_per_gpu=cfg.data.get('workers_per_gpu', 1),
+ dist=False,
+ shuffle=False,
+ pin_memory=False)
+ dataloader_setting = dict(dataloader_setting, **cfg.data.get('test_dataloader', {}))
+ data_loader = build_dataloader(dataset, **dataloader_setting)
+
+ model = MMDataParallel(model, device_ids=[0])
+ all_softmax, all_openmax, all_gts = [], [], []
+ prog_bar = mmcv.ProgressBar(len(data_loader.dataset))
+ for i, data in enumerate(data_loader):
+ with torch.no_grad():
+ feat_blob, score = model(return_loss=False, return_score=True, get_feat=True, **data)
+ softmax_prob = F.softmax(score, dim=1).cpu().numpy()
+ # aggregate features
+ feat_blob = spatial_temporal_pooling(feat_blob)
+ feat_final = feat_blob.cpu().numpy()
+ # Re-calibrate score before softmax with OpenMax
+ openmax_prob = openmax_recalibrate(weibull_model, feat_final, score.cpu().numpy())
+ # gather preds
+ all_openmax.append(openmax_prob)
+ all_softmax.append(softmax_prob)
+ # gather label
+ labels = data['label'].numpy()
+ all_gts.append(labels)
+
+ # use the first key as main key to calculate the batch size
+ batch_size = len(next(iter(data.values())))
+ for _ in range(batch_size):
+ prog_bar.update()
+
+ all_softmax = np.concatenate(all_softmax, axis=0)
+ all_openmax = np.concatenate(all_openmax, axis=0)
+ all_gts = np.concatenate(all_gts, axis=0)
+
+ return all_openmax, all_softmax, all_gts
+
+
+def evaluate_openmax(ind_openmax, ood_openmax, ind_labels, ood_labels, ood_ncls, num_rand=10):
+ ind_ncls = model.cls_head.num_classes
+ ind_results = np.argmax(ind_openmax, axis=1)
+ ood_results = np.argmax(ood_openmax, axis=1)
+
+ # close-set accuracy (multi-class)
+ acc = accuracy_score(ind_labels, ind_results)
+
+ # open-set auc-roc (binary class)
+ preds = np.concatenate((ind_results, ood_results), axis=0)
+ preds[preds == ind_ncls] = 1 # unknown class
+ preds[preds != 1] = 0 # known class
+ labels = np.concatenate((np.zeros_like(ind_labels), np.ones_like(ood_labels)))
+ auc = roc_auc_score(labels, preds)
+ print('OpenMax: ClosedSet Accuracy (multi-class): %.3lf, OpenSet AUC (bin-class): %.3lf'%(acc * 100, auc * 100))
+
+ # open set F1 score (multi-class)
+ macro_F1_list = [f1_score(ind_labels, ind_results, average='macro')]
+ std_list = [0]
+ openness_list = [0]
+ for n in range(ood_ncls):
+ ncls_novel = n + 1
+ openness = (1 - np.sqrt((2 * ind_ncls) / (2 * ind_ncls + ncls_novel))) * 100
+ openness_list.append(openness)
+ # randoml select the subset of ood samples
+ macro_F1_multi = np.zeros((num_rand), dtype=np.float32)
+ for m in range(num_rand):
+ cls_select = np.random.choice(ood_ncls, ncls_novel, replace=False)
+ ood_sub_results = np.concatenate([ood_results[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_labels = np.ones_like(ood_sub_results) * ind_ncls
+ # construct preds and labels
+ preds = np.concatenate((ind_results, ood_sub_results), axis=0)
+ labels = np.concatenate((ind_labels, ood_sub_labels), axis=0)
+ macro_F1_multi[m] = f1_score(labels, preds, average='macro')
+ macro_F1 = np.mean(macro_F1_multi)
+ std = np.std(macro_F1_multi)
+ macro_F1_list.append(macro_F1)
+ std_list.append(std)
+
+ # draw comparison curves
+ macro_F1_list = np.array(macro_F1_list)
+ std_list = np.array(std_list)
+
+ w_openness = np.array(openness_list) / 100.
+ open_maF1_mean = np.sum(w_openness * macro_F1_list) / np.sum(w_openness)
+ open_maF1_std = np.sum(w_openness * std_list) / np.sum(w_openness)
+ print('Open macro-F1 score: %.3f, std=%.3lf'%(open_maF1_mean * 100, open_maF1_std * 100))
+
+ return openness_list, macro_F1_list, std_list
+
+
+
+if __name__ == '__main__':
+ args = parse_args()
+ # assign the desired device.
+ device = torch.device(args.device)
+ set_deterministic(0)
+
+ # initialize recognition model
+ model = init_recognizer(args.config, args.checkpoint, device=device, use_frames=False)
+ torch.backends.cudnn.benchmark = True
+ model.cfg.data.test.test_mode = True
+
+ ######## Compute the Mean Activation Vector (MAV) and Distances ########
+ if not os.path.exists(args.cache_mav_dist):
+ os.makedirs(args.cache_mav_dist)
+ # parse the video files list of training set
+ videolist, labels = get_datalist(args.trainset_split)
+ # compute mav and dist
+ mav_dist_list = compute_mav_dist(videolist, labels, model, args.cache_mav_dist)
+
+
+ ######## OOD and IND detection ########
+ result_file = os.path.join(args.result_prefix + '_result.npz')
+ if not os.path.exists(result_file):
+ # prepare result path
+ result_dir = os.path.dirname(result_file)
+ if not os.path.exists(result_dir):
+ os.makedirs(result_dir)
+ # Weibull Model by EVT Fitting
+ print("Weibull fitting...")
+ weibull_model = weibull_fitting(mav_dist_list)
+ # run inference (OOD)
+ ood_openmax, ood_softmax, ood_labels = run_inference(model, weibull_model, args.ood_data)
+ # run inference (OOD)
+ ind_openmax, ind_softmax, ind_labels = run_inference(model, weibull_model, args.ind_data)
+ # save
+ np.savez(result_file[:-4], ind_openmax=ind_openmax, ood_openmax=ood_openmax,
+ ind_softmax=ind_softmax, ood_softmax=ood_softmax,
+ ind_label=ind_labels, ood_label=ood_labels)
+ else:
+ results = np.load(result_file, allow_pickle=True)
+ ind_openmax = results['ind_openmax'] # (N1, C+1)
+ ood_openmax = results['ood_openmax'] # (N2, C+1)
+ ind_softmax = results['ind_softmax'] # (N1, C)
+ ood_softmax = results['ood_softmax'] # (N2, C)
+ ind_labels = results['ind_label'] # (N1,)
+ ood_labels = results['ood_label'] # (N2,)
+
+ ######## Evaluation ########
+ openness_list, macro_F1_list, std_list = evaluate_openmax(ind_openmax, ood_openmax, ind_labels, ood_labels, args.ood_ncls, num_rand=args.num_rand)
+
+ # draw F1 curve
+ plt.figure(figsize=(8,5)) # (w, h)
+ plt.plot(openness_list, macro_F1_list, 'r-', linewidth=2)
+ # plt.fill_between(openness_list, macro_F1_list - std_list, macro_F1_list + std_list, 'c')
+ plt.ylim(0.5, 1.0)
+ plt.xlabel('Openness (%)')
+ plt.ylabel('macro F1')
+ plt.grid('on')
+ plt.legend('OpenMax')
+ plt.tight_layout()
+ dataset_name = args.result_prefix.split('_')[-1]
+ png_file = os.path.join(os.path.dirname(args.result_prefix), 'F1_openness_%s.png'%(dataset_name))
+ plt.savefig(png_file)
+ print('Openness curve figure is saved in: %s'%(png_file))
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/baseline_rpl.py b/Downstream/Open-Set-Action-Recognition/experiments/baseline_rpl.py
new file mode 100644
index 0000000..04c8246
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/baseline_rpl.py
@@ -0,0 +1,202 @@
+import argparse
+import os
+import os.path as osp
+import torch
+import mmcv
+from mmaction.apis import init_recognizer
+from mmcv.parallel import collate, scatter
+from mmaction.datasets.pipelines import Compose
+from mmaction.datasets import build_dataloader, build_dataset
+from mmcv.parallel import MMDataParallel
+import numpy as np
+from tqdm import tqdm
+from sklearn.metrics import f1_score, roc_auc_score, accuracy_score
+import torch.multiprocessing
+torch.multiprocessing.set_sharing_strategy('file_system')
+
+
+def parse_args():
+ """
+ experiments/baseline_rpl.py --config configs/recognition/tsm/inference_tsm_rpl.py \
+ --checkpoint work_dirs/tsm/finetune_ucf101_tsm_rpl/latest.pth \
+ --train_data data/ucf101/ucf101_train_split_1_videos.txt \
+ --ind_data
+ --result_prefix experiments/tsm/results_baselines/rpl/RPL
+ """
+ parser = argparse.ArgumentParser(description='MMAction2 test')
+ # model and data config
+ parser.add_argument('--config', help='test config file path')
+ parser.add_argument('--checkpoint', help='checkpoint file/url')
+ parser.add_argument('--train_data', help='the split file of in-distribution training data')
+ parser.add_argument('--batch_size', type=int, default=8, help='the testing batch size')
+ # test data config
+ parser.add_argument('--ind_data', help='the split file of in-distribution testing data')
+ parser.add_argument('--ood_data', help='the split file of out-of-distribution testing data')
+ parser.add_argument('--ood_ncls', type=int, help='the number of classes in unknwon dataset')
+ parser.add_argument('--ood_dataname', choices=['HMDB', 'MiT'], help='the name of out-of-distribution testing data')
+ # env config
+ parser.add_argument('--device', type=str, default='cuda:0', help='CPU/CUDA device option')
+ parser.add_argument('--result_prefix', help='result file prefix')
+ args = parser.parse_args()
+ return args
+
+
+def set_deterministic(seed):
+ torch.manual_seed(seed)
+ torch.cuda.manual_seed(seed)
+ torch.cuda.manual_seed_all(seed) # if you are using multi-GPU.
+ np.random.seed(seed) # Numpy module.
+ torch.backends.cudnn.benchmark = True
+ torch.backends.cudnn.deterministic = True
+
+
+def run_inference(config, checkpoint, data_split, batch_size, device):
+ # initialize recognition model
+ model = init_recognizer(config, checkpoint, device=device, use_frames=False)
+ torch.backends.cudnn.benchmark = True
+ model.cfg.data.test.test_mode = True
+ model.cfg.test_cfg.average_clips = 'prob' # we need the probability socore from softmax layer
+ model.cfg.data.videos_per_gpu = batch_size # batch size
+ model.cfg.data.test.ann_file = data_split
+ model.cfg.data.test.data_prefix = os.path.join(os.path.dirname(data_split), 'videos')
+
+ # build the dataloader
+ dataset = build_dataset(model.cfg.data.test, dict(test_mode=True))
+ dataloader_setting = dict(
+ videos_per_gpu=model.cfg.data.get('videos_per_gpu', 1),
+ workers_per_gpu=model.cfg.data.get('workers_per_gpu', 1),
+ dist=False,
+ shuffle=False,
+ pin_memory=False)
+ dataloader_setting = dict(dataloader_setting, **model.cfg.data.get('test_dataloader', {}))
+ data_loader = build_dataloader(dataset, **dataloader_setting)
+
+ # running the inference
+ model = MMDataParallel(model, device_ids=[0])
+ all_scores, all_labels = [], []
+ prog_bar = mmcv.ProgressBar(len(data_loader.dataset))
+ for i, data in enumerate(data_loader):
+ with torch.no_grad():
+ scores = model(return_loss=False, **data) # (B, C)
+ all_scores.append(scores)
+ # gather labels
+ labels = data['label'].numpy()
+ all_labels.append(labels)
+ # use the first key as main key to calculate the batch size
+ bs = len(next(iter(data.values())))
+ for _ in range(bs):
+ prog_bar.update()
+ all_scores = np.concatenate(all_scores, axis=0)
+ all_labels = np.concatenate(all_labels, axis=0)
+ return all_scores, all_labels
+
+
+def evaluate_softmax(ind_softmax, ood_softmax, ind_labels, ood_labels, ood_ncls, thresh, num_rand=10):
+
+ ind_ncls = ind_softmax.shape[1]
+ ind_results = np.argmax(ind_softmax, axis=1)
+ ood_results = np.argmax(ood_softmax, axis=1)
+ # close-set accuracy (multi-class)
+ acc = accuracy_score(ind_labels, ind_results)
+
+ # open-set auc-roc (binary class)
+ ind_conf = np.max(ind_softmax, axis=1)
+ ood_conf = np.max(ood_softmax, axis=1)
+ preds = np.concatenate((ind_results, ood_results), axis=0)
+ confs = np.concatenate((ind_conf, ood_conf), axis=0)
+ preds[confs < threshold] = 1 # unknown class
+ preds[confs >= threshold] = 0 # known class
+ labels = np.concatenate((np.zeros_like(ind_labels), np.ones_like(ood_labels)))
+ auc = roc_auc_score(labels, preds)
+ print('\nClosedSet Accuracy (multi-class): %.3lf, OpenSet AUC (bin-class): %.3lf'%(acc * 100, auc * 100))
+
+ ind_results[ind_conf < thresh] = ind_ncls # incorrect rejection
+ # open set F1 score (multi-class)
+ macro_F1_list = [f1_score(ind_labels, ind_results, average='macro')]
+ std_list = [0]
+ openness_list = [0]
+ for n in range(ood_ncls):
+ ncls_novel = n + 1
+ openness = (1 - np.sqrt((2 * ind_ncls) / (2 * ind_ncls + ncls_novel))) * 100
+ openness_list.append(openness)
+ # randoml select the subset of ood samples
+ macro_F1_multi = np.zeros((num_rand), dtype=np.float32)
+ for m in range(num_rand):
+ cls_select = np.random.choice(ood_ncls, ncls_novel, replace=False)
+ ood_sub_results = np.concatenate([ood_results[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_labels = np.ones_like(ood_sub_results) * ind_ncls
+ ood_sub_confs = np.concatenate([ood_conf[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_results[ood_sub_confs < thresh] = ind_ncls # correct rejection
+ # construct preds and labels
+ preds = np.concatenate((ind_results, ood_sub_results), axis=0)
+ labels = np.concatenate((ind_labels, ood_sub_labels), axis=0)
+ macro_F1_multi[m] = f1_score(labels, preds, average='macro')
+ macro_F1 = np.mean(macro_F1_multi)
+ std = np.std(macro_F1_multi)
+ macro_F1_list.append(macro_F1)
+ std_list.append(std)
+
+ # draw comparison curves
+ macro_F1_list = np.array(macro_F1_list)
+ std_list = np.array(std_list)
+
+ w_openness = np.array(openness_list) / 100.
+ open_maF1_mean = np.sum(w_openness * macro_F1_list) / np.sum(w_openness)
+ open_maF1_std = np.sum(w_openness * std_list) / np.sum(w_openness)
+ print('Open macro-F1 score: %.3f, std=%.3lf'%(open_maF1_mean * 100, open_maF1_std * 100))
+
+ return openness_list, macro_F1_list, std_list
+
+
+if __name__ == '__main__':
+
+ args = parse_args()
+ # assign the desired device.
+ device = torch.device(args.device)
+ set_deterministic(0)
+ modelname = os.path.dirname(args.config).split('/')[-1].upper()
+
+ ######## Compute threshold with training data ########
+ result_file = os.path.join(os.path.dirname(args.result_prefix), modelname + '_RPL_trainset_softmax.npz')
+ if not os.path.exists(result_file):
+ # prepare result path
+ result_dir = os.path.dirname(result_file)
+ if not os.path.exists(result_dir):
+ os.makedirs(result_dir)
+ # run the inference on training data
+ trainset_scores, _ = run_inference(args.config, args.checkpoint, args.train_data, args.batch_size, device)
+ # save
+ np.savez(result_file[:-4], trainset_scores=trainset_scores)
+ else:
+ result = np.load(result_file)
+ trainset_scores = result['trainset_scores']
+
+ max_scores = np.max(trainset_scores, axis=1)
+ scores_sort = np.sort(max_scores)[::-1] # sort the uncertainties with descending order
+ N = max_scores.shape[0]
+ threshold = scores_sort[int(N * 0.95)-1] # 95% percentile
+
+ print('\nThe RPL softmax threshold on UCF-101 train set: %lf'%(threshold))
+
+ ######## OOD and IND detection ########
+ testset_result = os.path.join(os.path.dirname(args.result_prefix), modelname +'_RPL_'+ args.ood_dataname +'_result.npz')
+ if not os.path.exists(testset_result):
+ # prepare result path
+ result_dir = os.path.dirname(testset_result)
+ if not os.path.exists(result_dir):
+ os.makedirs(result_dir)
+ # run the inference on OOD data
+ ood_softmax, ood_labels = run_inference(args.config, args.checkpoint, args.ood_data, args.batch_size, device)
+ # run the inference on IND data
+ ind_softmax, ind_labels = run_inference(args.config, args.checkpoint, args.ind_data, args.batch_size, device)
+ # save
+ np.savez(testset_result[:-4], ind_softmax=ind_softmax, ood_softmax=ood_softmax,
+ ind_label=ind_labels, ood_label=ood_labels)
+ else:
+ results = np.load(testset_result, allow_pickle=True)
+ ind_softmax = results['ind_softmax'] # (N1, C)
+ ood_softmax = results['ood_softmax'] # (N2, C)
+ ind_labels = results['ind_label'] # (N1,)
+ ood_labels = results['ood_label'] # (N2,)
+
+ openness_list, macro_F1_list, std_list = evaluate_softmax(ind_softmax, ood_softmax, ind_labels, ood_labels, args.ood_ncls, threshold)
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/baseline_softmax.py b/Downstream/Open-Set-Action-Recognition/experiments/baseline_softmax.py
new file mode 100644
index 0000000..dd5084f
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/baseline_softmax.py
@@ -0,0 +1,182 @@
+import argparse, os, sys
+import torch
+import mmcv
+import numpy as np
+import torch.nn.functional as F
+from mmcv.parallel import collate, scatter
+from mmaction.datasets.pipelines import Compose
+from mmaction.apis import init_recognizer
+from mmaction.datasets import build_dataloader, build_dataset
+from mmcv.parallel import MMDataParallel
+from tqdm import tqdm
+from sklearn.metrics import f1_score, roc_auc_score, accuracy_score
+import matplotlib.pyplot as plt
+
+
+def parse_args():
+ """ Example shell script:
+ $ cd experiments
+ $ source activate mmaction
+ $ nohup python baseline_openmax.py --model i3d --config configs/recognition/tpn/inference_tpn_slowonly_dnn.py
+ """
+ parser = argparse.ArgumentParser(description='MMAction2 test')
+ # model config
+ parser.add_argument('--config', help='test config file path')
+ parser.add_argument('--checkpoint', help='checkpoint file/url')
+ parser.add_argument('--train_data', default='data/ucf101/ucf101_train_split_1_videos.txt', help='the split file of in-distribution training data')
+ parser.add_argument('--batch_size', type=int, default=8, help='the testing batch size')
+ parser.add_argument('--ood_ncls', type=int, help='the number of classes in unknwon dataset')
+ # device
+ parser.add_argument('--device', type=str, default='cuda:0', help='CPU/CUDA device option')
+ parser.add_argument('--result_prefix', help='result file prefix')
+ args = parser.parse_args()
+ return args
+
+
+def set_deterministic(seed):
+ torch.manual_seed(seed)
+ torch.cuda.manual_seed(seed)
+ torch.cuda.manual_seed_all(seed) # if you are using multi-GPU.
+ np.random.seed(seed) # Numpy module.
+ torch.backends.cudnn.benchmark = True
+ torch.backends.cudnn.deterministic = True
+
+
+def run_inference(config, checkpoint, train_data, batch_size, device):
+ # initialize recognition model
+ model = init_recognizer(config, checkpoint, device=device, use_frames=False)
+ torch.backends.cudnn.benchmark = True
+ model.cfg.data.test.test_mode = True
+ model.cfg.test_cfg.average_clips = 'prob' # we need the probability socore from softmax layer
+ model.cfg.data.videos_per_gpu = batch_size # batch size of training set
+ # We use training data to obtain the threshold
+ model.cfg.data.test.ann_file = train_data
+ model.cfg.data.test.data_prefix = os.path.join(os.path.dirname(train_data), 'videos')
+
+ # build the dataloader
+ dataset = build_dataset(model.cfg.data.test, dict(test_mode=True))
+ dataloader_setting = dict(
+ videos_per_gpu=model.cfg.data.get('videos_per_gpu', 1),
+ workers_per_gpu=model.cfg.data.get('workers_per_gpu', 1),
+ dist=False,
+ shuffle=False,
+ pin_memory=False)
+ dataloader_setting = dict(dataloader_setting, **model.cfg.data.get('test_dataloader', {}))
+ data_loader = build_dataloader(dataset, **dataloader_setting)
+
+ # running the inference
+ model = MMDataParallel(model, device_ids=[0])
+ all_scores = []
+ prog_bar = mmcv.ProgressBar(len(data_loader.dataset))
+ for i, data in enumerate(data_loader):
+ with torch.no_grad():
+ scores = model(return_loss=False, **data) # (B, C)
+ all_scores.append(scores)
+ # use the first key as main key to calculate the batch size
+ bs = len(next(iter(data.values())))
+ for _ in range(bs):
+ prog_bar.update()
+ all_scores = np.concatenate(all_scores, axis=0)
+ return all_scores
+
+
+def evaluate_softmax(ind_softmax, ood_softmax, ind_labels, ood_labels, ood_ncls, thresh, num_rand=10):
+
+ ind_ncls = ind_softmax.shape[1]
+ ind_results = np.argmax(ind_softmax, axis=1)
+ ood_results = np.argmax(ood_softmax, axis=1)
+ # close-set accuracy (multi-class)
+ acc = accuracy_score(ind_labels, ind_results)
+
+ # open-set auc-roc (binary class)
+ ind_conf = np.max(ind_softmax, axis=1)
+ ood_conf = np.max(ood_softmax, axis=1)
+ preds = np.concatenate((ind_results, ood_results), axis=0)
+ confs = np.concatenate((ind_conf, ood_conf), axis=0)
+ preds[confs < threshold] = 1 # unknown class
+ preds[confs >= threshold] = 0 # known class
+ labels = np.concatenate((np.zeros_like(ind_labels), np.ones_like(ood_labels)))
+ auc = roc_auc_score(labels, preds)
+ print('SoftMax: ClosedSet Accuracy (multi-class): %.3lf, OpenSet AUC (bin-class): %.3lf'%(acc * 100, auc * 100))
+
+ ind_results[ind_conf < thresh] = ind_ncls # incorrect rejection
+ # open set F1 score (multi-class)
+ macro_F1_list = [f1_score(ind_labels, ind_results, average='macro')]
+ std_list = [0]
+ openness_list = [0]
+ for n in range(ood_ncls):
+ ncls_novel = n + 1
+ openness = (1 - np.sqrt((2 * ind_ncls) / (2 * ind_ncls + ncls_novel))) * 100
+ openness_list.append(openness)
+ # randoml select the subset of ood samples
+ macro_F1_multi = np.zeros((num_rand), dtype=np.float32)
+ for m in range(num_rand):
+ cls_select = np.random.choice(ood_ncls, ncls_novel, replace=False)
+ ood_sub_results = np.concatenate([ood_results[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_labels = np.ones_like(ood_sub_results) * ind_ncls
+ ood_sub_confs = np.concatenate([ood_conf[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_results[ood_sub_confs < thresh] = ind_ncls # correct rejection
+ # construct preds and labels
+ preds = np.concatenate((ind_results, ood_sub_results), axis=0)
+ labels = np.concatenate((ind_labels, ood_sub_labels), axis=0)
+ macro_F1_multi[m] = f1_score(labels, preds, average='macro')
+ macro_F1 = np.mean(macro_F1_multi)
+ std = np.std(macro_F1_multi)
+ macro_F1_list.append(macro_F1)
+ std_list.append(std)
+
+ # draw comparison curves
+ macro_F1_list = np.array(macro_F1_list)
+ std_list = np.array(std_list)
+
+ w_openness = np.array(openness_list) / 100.
+ open_maF1_mean = np.sum(w_openness * macro_F1_list) / np.sum(w_openness)
+ open_maF1_std = np.sum(w_openness * std_list) / np.sum(w_openness)
+ print('Open macro-F1 score: %.3f, std=%.3lf'%(open_maF1_mean * 100, open_maF1_std * 100))
+
+ return openness_list, macro_F1_list, std_list
+
+
+if __name__ == '__main__':
+ args = parse_args()
+ # assign the desired device.
+ device = torch.device(args.device)
+ set_deterministic(0)
+ modelname, methodname, dataname = args.result_prefix.split('/')[-1].split('_')
+
+ ######## Compute threshold with training data ########
+ result_file = os.path.join(os.path.dirname(args.result_prefix), modelname +'_SoftMax_trainset_softmax.npz')
+ if not os.path.exists(result_file):
+ # prepare result path
+ result_dir = os.path.dirname(result_file)
+ if not os.path.exists(result_dir):
+ os.makedirs(result_dir)
+ # run the inference on training data
+ trainset_scores = run_inference(args.config, args.checkpoint, args.train_data, args.batch_size, device)
+ # save
+ np.savez(result_file[:-4], trainset_scores=trainset_scores)
+ else:
+ result = np.load(result_file)
+ trainset_scores = result['trainset_scores']
+
+ max_scores = np.max(trainset_scores, axis=1)
+ scores_sort = np.sort(max_scores)[::-1] # sort the uncertainties with descending order
+ N = max_scores.shape[0]
+ threshold = scores_sort[int(N * 0.95)-1]
+
+ print('\nThe model %s softmax threshold on UCF-101 train set: %lf'%(args.result_prefix.split('/')[-1], threshold))
+
+ # load the softmax results on testing dataset (in OpenMax baseline)
+ ######## OOD and IND detection ########
+ openmax_result = os.path.join(os.path.dirname(args.result_prefix), '../openmax', modelname +'_OpenMax_'+ dataname +'_result.npz')
+ if not os.path.exists(openmax_result):
+ print('File does not exist! %s'%(openmax_result))
+ print('Run baseline_openmax.py first to get softmax testing results!')
+ else:
+ results = np.load(openmax_result, allow_pickle=True)
+ ind_softmax = results['ind_softmax'] # (N1, C)
+ ood_softmax = results['ood_softmax'] # (N2, C)
+ ind_labels = results['ind_label'] # (N1,)
+ ood_labels = results['ood_label'] # (N2,)
+
+ openness_list, macro_F1_list, std_list = evaluate_softmax(ind_softmax, ood_softmax, ind_labels, ood_labels, args.ood_ncls, threshold)
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/compare_openness.py b/Downstream/Open-Set-Action-Recognition/experiments/compare_openness.py
new file mode 100644
index 0000000..6a51dcd
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/compare_openness.py
@@ -0,0 +1,114 @@
+import os
+import argparse
+import numpy as np
+from sklearn.metrics import f1_score, roc_auc_score, accuracy_score
+import matplotlib.pyplot as plt
+
+def parse_args():
+ '''Command instruction:
+ source activate mmaction
+ python experiments/compare_openness.py --ind_ncls 101 --ood_ncls 51
+ '''
+ parser = argparse.ArgumentParser(description='Compare the performance of openness')
+ # model config
+ parser.add_argument('--base_model', default='i3d', help='the backbone model name')
+ parser.add_argument('--baselines', nargs='+', default=['I3D_Dropout_BALD', 'I3D_BNN_BALD', 'I3D_EDLlog_EDL', 'I3D_EDLlogAvUC_EDL'])
+ parser.add_argument('--thresholds', nargs='+', type=float, default=[0.000423, 0.000024, 0.495783, 0.495783])
+ parser.add_argument('--styles', nargs='+', default=['-b', '-k', '-r', '-g', '-m'])
+ parser.add_argument('--ind_ncls', type=int, default=101, help='the number of classes in known dataset')
+ parser.add_argument('--ood_ncls', type=int, help='the number of classes in unknwon dataset')
+ parser.add_argument('--ood_data', default='HMDB', help='the name of OOD dataset.')
+ parser.add_argument('--num_rand', type=int, default=10, help='the number of random selection for ood classes')
+ parser.add_argument('--result_png', default='F1_openness_compare_HMDB.png')
+ args = parser.parse_args()
+ return args
+
+
+def main():
+
+ result_path = os.path.join('./experiments', args.base_model, 'results')
+ plt.figure(figsize=(8,5)) # (w, h)
+ plt.rcParams["font.family"] = "Arial" # Times New Roman
+ fontsize = 15
+ for style, thresh, baseline in zip(args.styles, args.thresholds, args.baselines):
+ result_file = os.path.join(result_path, baseline + '_%s'%(args.ood_data) + '_result.npz')
+ assert os.path.exists(result_file), "File not found! Run ood_detection first!"
+ # load the testing results
+ results = np.load(result_file, allow_pickle=True)
+ ind_uncertainties = results['ind_unctt'] # (N1,)
+ ood_uncertainties = results['ood_unctt'] # (N2,)
+ ind_results = results['ind_pred'] # (N1,)
+ ood_results = results['ood_pred'] # (N2,)
+ ind_labels = results['ind_label']
+ ood_labels = results['ood_label']
+
+ # close-set accuracy (multi-class)
+ acc = accuracy_score(ind_labels, ind_results)
+ # open-set auc-roc (binary class)
+ preds = np.concatenate((ind_results, ood_results), axis=0)
+ uncertains = np.concatenate((ind_uncertainties, ood_uncertainties), axis=0)
+ preds[uncertains > thresh] = 1
+ preds[uncertains <= thresh] = 0
+ labels = np.concatenate((np.zeros_like(ind_labels), np.ones_like(ood_labels)))
+ aupr = roc_auc_score(labels, preds)
+ print('Model: %s, ClosedSet Accuracy (multi-class): %.3lf, OpenSet AUC (bin-class): %.3lf'%(baseline, acc * 100, aupr * 100))
+
+ # open set F1 score (multi-class)
+ ind_results[ind_uncertainties > thresh] = args.ind_ncls # falsely rejection
+ macro_F1_list = [f1_score(ind_labels, ind_results, average='macro')]
+ std_list = [0]
+ openness_list = [0]
+ for n in range(args.ood_ncls):
+ ncls_novel = n + 1
+ openness = (1 - np.sqrt((2 * args.ind_ncls) / (2 * args.ind_ncls + ncls_novel))) * 100
+ openness_list.append(openness)
+ # randoml select the subset of ood samples
+ macro_F1_multi = np.zeros((args.num_rand), dtype=np.float32)
+ for m in range(args.num_rand):
+ cls_select = np.random.choice(args.ood_ncls, ncls_novel, replace=False)
+ ood_sub_results = np.concatenate([ood_results[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_uncertainties = np.concatenate([ood_uncertainties[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_results[ood_sub_uncertainties > thresh] = args.ind_ncls # correctly rejection
+ ood_sub_labels = np.ones_like(ood_sub_results) * args.ind_ncls
+ # construct preds and labels
+ preds = np.concatenate((ind_results, ood_sub_results), axis=0)
+ labels = np.concatenate((ind_labels, ood_sub_labels), axis=0)
+ macro_F1_multi[m] = f1_score(labels, preds, average='macro')
+ macro_F1 = np.mean(macro_F1_multi)
+ std = np.std(macro_F1_multi)
+ macro_F1_list.append(macro_F1)
+ std_list.append(std)
+
+ # draw comparison curves
+ macro_F1_list = np.array(macro_F1_list)
+ std_list = np.array(std_list)
+ plt.plot(openness_list, macro_F1_list * 100, style, linewidth=2)
+ # plt.fill_between(openness_list, macro_F1_list - std_list, macro_F1_list + std_list, style)
+
+ w_openness = np.array(openness_list) / 100.
+ open_maF1_mean = np.sum(w_openness * macro_F1_list) / np.sum(w_openness)
+ open_maF1_std = np.sum(w_openness * std_list) / np.sum(w_openness)
+ print('Open macro-F1 score: %.3f, std=%.3lf'%(open_maF1_mean * 100, open_maF1_std * 100))
+
+ plt.xlim(0, max(openness_list))
+ plt.ylim(60, 80)
+ plt.xlabel('Openness (%)', fontsize=fontsize)
+ plt.ylabel('macro F1 (%)', fontsize=fontsize)
+ plt.grid('on')
+ # plt.legend(args.baselines)
+ plt.legend(['MC Dropout BALD', 'BNN SVI BALD', 'DEAR (vanilla)', 'DEAR (alter)', 'DEAR (joint)'], loc='lower left', fontsize=fontsize)
+ plt.xticks(fontsize=fontsize)
+ plt.yticks(fontsize=fontsize)
+ plt.tight_layout()
+ png_file = os.path.join(result_path, args.result_png)
+ plt.savefig(png_file)
+ plt.savefig(png_file[:-4] + '.pdf')
+ print('Openness curve figure is saved in: %s'%(png_file))
+
+
+if __name__ == "__main__":
+
+ np.random.seed(123)
+ args = parse_args()
+
+ main()
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/compare_openness_new.py b/Downstream/Open-Set-Action-Recognition/experiments/compare_openness_new.py
new file mode 100644
index 0000000..4a12cbf
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/compare_openness_new.py
@@ -0,0 +1,167 @@
+import os
+import argparse
+from matplotlib.pyplot import axis
+import numpy as np
+from sklearn.metrics import roc_auc_score, accuracy_score, precision_recall_curve, auc, roc_curve
+from terminaltables import AsciiTable
+
+def parse_args():
+ '''Command instruction:
+ source activate mmaction
+ python experiments/compare_openness.py
+ '''
+ parser = argparse.ArgumentParser(description='Compare the performance of openness')
+ # model config
+ parser.add_argument('--base_model', default='i3d', help='the backbone model name')
+ parser.add_argument('--ood_data', default='HMDB', help='the name of OOD dataset.')
+ parser.add_argument('--thresholds', nargs='+', type=float, default=[-1,-1,-1,-1,-1,-1])
+ parser.add_argument('--baseline_results', nargs='+', help='the testing results files.')
+ args = parser.parse_args()
+ return args
+
+
+def eval_osr(y_true, y_pred):
+ # open-set auc-roc (binary class)
+ auroc = roc_auc_score(y_true, y_pred)
+
+ # open-set auc-pr (binary class)
+ # as an alternative, you may also use `ap = average_precision_score(labels, uncertains)`, which is approximate to aupr.
+ precision, recall, _ = precision_recall_curve(y_true, y_pred)
+ aupr = auc(recall, precision)
+
+ # open-set fpr@95 (binary class)
+ fpr, tpr, _ = roc_curve(y_true, y_pred, pos_label=1)
+ operation_idx = np.abs(tpr - 0.95).argmin()
+ fpr95 = fpr[operation_idx] # FPR when TPR at 95%
+
+ return auroc, aupr, fpr95
+
+
+def parse_results(result_file, method='softmax'):
+ # Softmax and OpenMax
+ assert os.path.exists(result_file), "File not found! Run baseline_openmax.py first to get softmax testing results!\n%s"%(result_file)
+ results = np.load(result_file, allow_pickle=True)
+ # parse results
+ ind_labels = results['ind_label'] # (N1,)
+ ood_labels = results['ood_label'] # (N2,)
+ if method == 'softmax':
+ ind_softmax = results['ind_softmax'] # (N1, C)
+ ood_softmax = results['ood_softmax'] # (N2, C)
+ return ind_softmax, ood_softmax, ind_labels, ood_labels
+ elif method == 'openmax':
+ ind_openmax = results['ind_openmax'] # (N1, C+1)
+ ood_openmax = results['ood_openmax'] # (N2, C+1)
+ return ind_openmax, ood_openmax, ind_labels, ood_labels
+
+
+def eval_confidence_methods(ind_probs, ood_probs, ind_labels, ood_labels, score='max_prob', ind_ncls=101, threshold=-1):
+ # close-set accuracy (multi-class)
+ ind_results = np.argmax(ind_probs, axis=1)
+ ood_results = np.argmax(ood_probs, axis=1)
+ acc = accuracy_score(ind_labels, ind_results)
+
+ # open-set evaluation (binary class)
+ if score == 'binary':
+ preds = np.concatenate((ind_results, ood_results), axis=0)
+ idx_pos = preds == ind_ncls
+ idx_neg = preds != ind_ncls
+ preds[idx_pos] = 1 # unknown class
+ preds[idx_neg] = 0 # known class
+ elif score == 'max_prob':
+ ind_conf = np.max(ind_probs, axis=1)
+ ood_conf = np.max(ood_probs, axis=1)
+ confs = np.concatenate((ind_conf, ood_conf), axis=0)
+ if threshold > 0:
+ preds = np.concatenate((ind_results, ood_results), axis=0)
+ preds[confs < threshold] = 1 # unknown class
+ preds[confs >= threshold] = 0 # known class
+ else:
+ preds = 1 - confs
+ labels = np.concatenate((np.zeros_like(ind_labels), np.ones_like(ood_labels)))
+ auroc, aupr, fpr95 = eval_osr(labels, preds)
+
+ return acc, auroc, aupr, fpr95
+
+
+def eval_uncertainty_methods(result_file, threshold=-1):
+ assert os.path.exists(result_file), "File not found! Run ood_detection first!\n%s"%(result_file)
+ # load the testing results
+ results = np.load(result_file, allow_pickle=True)
+ ind_uncertainties = results['ind_unctt'] # (N1,)
+ ood_uncertainties = results['ood_unctt'] # (N2,)
+ ind_results = results['ind_pred'] # (N1,)
+ ood_results = results['ood_pred'] # (N2,)
+ ind_labels = results['ind_label']
+ ood_labels = results['ood_label']
+
+ # close-set accuracy (multi-class)
+ acc = accuracy_score(ind_labels, ind_results)
+
+ # open-set evaluation (binary class)
+ if threshold > 0:
+ preds = np.concatenate((ind_results, ood_results), axis=0)
+ uncertains = np.concatenate((ind_uncertainties, ood_uncertainties), axis=0)
+ preds[uncertains > threshold] = 1
+ preds[uncertains <= threshold] = 0
+ else:
+ preds = np.concatenate((ind_uncertainties, ood_uncertainties), axis=0)
+
+ labels = np.concatenate((np.zeros_like(ind_labels), np.ones_like(ood_labels)))
+ auroc, aupr, fpr95 = eval_osr(labels, preds)
+
+ return acc, auroc, aupr, fpr95
+
+
+def main():
+
+ print(f'\nNew Evaluation Results (open-set data: {args.ood_data}, backbone: {args.base_model})')
+ display_data = [["Methods", "AUROC (%)", "AUPR (%)", "FPR@95 (%)", "Closed-Set ACC (%)"],
+ ["OpenMax"], ["MC Dropout"], ["BNN SVI"], ["SoftMax"], ["RPL"], ["DEAR (ours)"]] # table heads and rows
+ exp_dir = os.path.join('./experiments', args.base_model)
+
+ # OpenMax
+ result_path = os.path.join(exp_dir, args.baseline_results[0])
+ ind_openmax, ood_openmax, ind_labels, ood_labels = parse_results(result_path, method='openmax')
+ acc, auroc, aupr, fpr95 = eval_confidence_methods(ind_openmax, ood_openmax, ind_labels, ood_labels, score='binary')
+ display_data[1].extend(["%.3f"%(auroc * 100), "%.3f"%(aupr * 100), "%.3f"%(fpr95 * 100), "%.3f"%(acc * 100)])
+
+ # MC Dropout
+ result_path = os.path.join(exp_dir, args.baseline_results[1])
+ acc, auroc, aupr, fpr95 = eval_uncertainty_methods(result_path, threshold=args.thresholds[1])
+ display_data[2].extend(["%.3f"%(auroc * 100), "%.3f"%(aupr * 100), "%.3f"%(fpr95 * 100), "%.3f"%(acc * 100)])
+
+ # BNN SVI
+ result_path = os.path.join(exp_dir, args.baseline_results[2])
+ acc, auroc, aupr, fpr95 = eval_uncertainty_methods(result_path, threshold=args.thresholds[2])
+ display_data[3].extend(["%.3f"%(auroc * 100), "%.3f"%(aupr * 100), "%.3f"%(fpr95 * 100), "%.3f"%(acc * 100)])
+
+ # SoftMax
+ result_path = os.path.join(exp_dir, args.baseline_results[3])
+ ind_softmax, ood_softmax, ind_labels, ood_labels = parse_results(result_path, method='softmax')
+ acc, auroc, aupr, fpr95 = eval_confidence_methods(ind_softmax, ood_softmax, ind_labels, ood_labels, threshold=args.thresholds[3])
+ display_data[4].extend(["%.3f"%(auroc * 100), "%.3f"%(aupr * 100), "%.3f"%(fpr95 * 100), "%.3f"%(acc * 100)])
+
+ # RPL
+ result_path = os.path.join(exp_dir, args.baseline_results[4])
+ ind_softmax, ood_softmax, ind_labels, ood_labels = parse_results(result_path, method='softmax')
+ acc, auroc, aupr, fpr95 = eval_confidence_methods(ind_softmax, ood_softmax, ind_labels, ood_labels, threshold=args.thresholds[4])
+ display_data[5].extend(["%.3f"%(auroc * 100), "%.3f"%(aupr * 100), "%.3f"%(fpr95 * 100), "%.3f"%(acc * 100)])
+
+ # DEAR (ours)
+ result_path = os.path.join(exp_dir, args.baseline_results[5])
+ acc, auroc, aupr, fpr95 = eval_uncertainty_methods(result_path, threshold=args.thresholds[5])
+ display_data[6].extend(["%.3f"%(auroc * 100), "%.3f"%(aupr * 100), "%.3f"%(fpr95 * 100), "%.3f"%(acc * 100)])
+
+
+ table = AsciiTable(display_data)
+ table.inner_footing_row_border = True
+ table.justify_columns = {0: 'left', 1: 'center', 2: 'center', 3: 'center', 4: 'center'}
+ print(table.table)
+ print("\n")
+
+if __name__ == "__main__":
+
+ np.random.seed(123)
+ args = parse_args()
+
+ main()
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/csn/baseline_csn_openmax.sh b/Downstream/Open-Set-Action-Recognition/experiments/csn/baseline_csn_openmax.sh
new file mode 100644
index 0000000..385f5d4
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/csn/baseline_csn_openmax.sh
@@ -0,0 +1,40 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+DEVICE=$1
+OOD_DATASET=$2
+IND_DATA='data/ucf101/ucf101_val_split_1_videos.txt'
+
+case ${OOD_DATASET} in
+ HMDB)
+ # run ood detection on hmdb-51 validation set
+ OOD_DATA='data/hmdb51/hmdb51_val_split_1_videos.txt'
+ NUM_CLASSES=51
+ ;;
+ MiT)
+ # run ood detection on mit-v2 validation set
+ OOD_DATA='data/mit/mit_val_list_videos.txt'
+ NUM_CLASSES=305
+ ;;
+ *)
+ echo "Dataset not supported: "${OOD_DATASET}
+ exit
+ ;;
+esac
+RESULT_DIR='experiments/csn/results'
+
+CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/baseline_openmax.py \
+ --config configs/recognition/csn/inference_csn_dnn.py \
+ --checkpoint work_dirs/csn/finetune_ucf101_csn_dnn/latest.pth \
+ --cache_mav_dist experiments/csn/results_baselines/openmax/ucf101_mav_dist \
+ --ind_data ${IND_DATA} \
+ --ood_data ${OOD_DATA} \
+ --ood_ncls ${NUM_CLASSES} \
+ --result_prefix experiments/csn/results_baselines/openmax/CSN_OpenMax_${OOD_DATASET}
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/csn/baseline_csn_softmax.sh b/Downstream/Open-Set-Action-Recognition/experiments/csn/baseline_csn_softmax.sh
new file mode 100644
index 0000000..9dccbe8
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/csn/baseline_csn_softmax.sh
@@ -0,0 +1,33 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+DEVICE=$1
+OOD_DATASET=$2
+
+case ${OOD_DATASET} in
+ HMDB)
+ NUM_CLASSES=51
+ ;;
+ MiT)
+ NUM_CLASSES=305
+ ;;
+ *)
+ echo "Dataset not supported: "${OOD_DATASET}
+ exit
+ ;;
+esac
+RESULT_DIR='experiments/csn/results'
+
+CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/baseline_softmax.py \
+ --config configs/recognition/csn/inference_csn_dnn.py \
+ --checkpoint work_dirs/csn/finetune_ucf101_csn_dnn/latest.pth \
+ --batch_size 1 \
+ --ood_ncls ${NUM_CLASSES} \
+ --result_prefix experiments/csn/results_baselines/softmax/CSN_SoftMax_${OOD_DATASET}
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/csn/evaluate_csn_dnn_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/csn/evaluate_csn_dnn_ucf101.sh
new file mode 100644
index 0000000..9174858
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/csn/evaluate_csn_dnn_ucf101.sh
@@ -0,0 +1,15 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+CUDA_VISIBLE_DEVICES=$1 python tools/test.py configs/recognition/csn/finetune_ucf101_csn_dnn.py \
+ work_dirs/csn/finetune_ucf101_csn_dnn/latest.pth \
+ --videos_per_gpu 1 \
+ --out work_dirs/csn/test_ucf101_csn_dnn.pkl \
+ --eval top_k_accuracy mean_class_accuracy
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/csn/evaluate_csn_edlnokl_avuc_debias_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/csn/evaluate_csn_edlnokl_avuc_debias_ucf101.sh
new file mode 100644
index 0000000..e5fd553
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/csn/evaluate_csn_edlnokl_avuc_debias_ucf101.sh
@@ -0,0 +1,15 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+CUDA_VISIBLE_DEVICES=$1 python tools/test.py configs/recognition/csn/finetune_ucf101_csn_edlnokl_avuc_debias.py \
+ work_dirs/csn/finetune_ucf101_csn_edlnokl_avuc_debias/latest.pth \
+ --videos_per_gpu 1 \
+ --out work_dirs/csn/test_ucf101_csn_edlnokl_avuc_debias.pkl \
+ --eval top_k_accuracy mean_class_accuracy
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/csn/finetune_csn_dnn_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/csn/finetune_csn_dnn_ucf101.sh
new file mode 100644
index 0000000..8bad83c
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/csn/finetune_csn_dnn_ucf101.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+# --validate
+CUDA_VISIBLE_DEVICES=$1 python tools/train.py configs/recognition/csn/finetune_ucf101_csn_dnn.py \
+ --work-dir work_dirs/csn/finetune_ucf101_csn_dnn \
+ --seed 0 \
+ --deterministic \
+ --gpu-ids 0 \
+ --validate
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/csn/finetune_csn_edlnokl_avuc_debias_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/csn/finetune_csn_edlnokl_avuc_debias_ucf101.sh
new file mode 100644
index 0000000..c274fb2
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/csn/finetune_csn_edlnokl_avuc_debias_ucf101.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+# --validate
+CUDA_VISIBLE_DEVICES=$1 python tools/train.py configs/recognition/csn/finetune_ucf101_csn_edlnokl_avuc_debias.py \
+ --work-dir work_dirs/csn/finetune_ucf101_csn_edlnokl_avuc_debias \
+ --seed 0 \
+ --deterministic \
+ --gpu-ids 0 \
+ --validate
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/csn/run_get_threshold.sh b/Downstream/Open-Set-Action-Recognition/experiments/csn/run_get_threshold.sh
new file mode 100644
index 0000000..f578c1e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/csn/run_get_threshold.sh
@@ -0,0 +1,45 @@
+#!/bin/bash
+
+export CUDA_HOME='/usr/local/cuda'
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+DEVICE=$1
+MODEL=$2
+BATCHSIZE=$3
+TRAIN_DATA='data/ucf101/ucf101_train_split_1_videos.txt'
+RESULT_DIR='experiments/csn/results'
+
+case ${MODEL} in
+ dnn)
+ # get the BALD threshold for csn model trained on UCF-101
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/get_threshold.py \
+ --config configs/recognition/csn/inference_csn_dnn.py \
+ --checkpoint work_dirs/csn/finetune_ucf101_csn_dnn/latest.pth \
+ --train_data ${TRAIN_DATA} \
+ --batch_size ${BATCHSIZE} \
+ --uncertainty BALD \
+ --result_prefix ${RESULT_DIR}/CSN_DNN_BALD
+ ;;
+ edlnokl_avuc_debias)
+ # get the EDL threshold for CSN_EDL_AvUC_Debias model trained on UCF-101
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/get_threshold.py \
+ --config configs/recognition/csn/inference_csn_enn.py \
+ --checkpoint work_dirs/csn/finetune_ucf101_csn_edlnokl_avuc_debias/latest.pth \
+ --train_data ${TRAIN_DATA} \
+ --batch_size ${BATCHSIZE} \
+ --uncertainty EDL \
+ --result_prefix ${RESULT_DIR}/CSN_EDLNoKLAvUCDebias_EDL
+ ;;
+ *)
+ echo "Invalid model: "${MODEL}
+ exit
+ ;;
+esac
+
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/csn/run_ood_detection.sh b/Downstream/Open-Set-Action-Recognition/experiments/csn/run_ood_detection.sh
new file mode 100644
index 0000000..7ce5f0e
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/csn/run_ood_detection.sh
@@ -0,0 +1,60 @@
+#!/bin/bash
+
+export CUDA_HOME='/usr/local/cuda'
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+DEVICE=$1
+OOD_DATASET=$2
+MODEL=$3
+IND_DATA='data/ucf101/ucf101_val_split_1_videos.txt'
+
+case ${OOD_DATASET} in
+ HMDB)
+ # run ood detection on hmdb-51 validation set
+ OOD_DATA='data/hmdb51/hmdb51_val_split_1_videos.txt'
+ ;;
+ MiT)
+ # run ood detection on hmdb-51 validation set
+ OOD_DATA='data/mit/mit_val_list_videos.txt'
+ ;;
+ *)
+ echo "Dataset not supported: "${OOD_DATASET}
+ exit
+ ;;
+esac
+RESULT_DIR='experiments/csn/results'
+
+case ${MODEL} in
+ dnn)
+ # DNN with Dropout model
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/ood_detection.py \
+ --config configs/recognition/csn/inference_csn_dnn.py \
+ --checkpoint work_dirs/csn/finetune_ucf101_csn_dnn/latest.pth \
+ --ind_data ${IND_DATA} \
+ --ood_data ${OOD_DATA} \
+ --uncertainty BALD \
+ --result_prefix ${RESULT_DIR}/CSN_DNN_BALD_${OOD_DATASET}
+ ;;
+ edlnokl_avuc_debias)
+ # Evidential Deep Learning (without KL divergence loss term) with AvU Calibration and Debiasing
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/ood_detection.py \
+ --config configs/recognition/csn/inference_csn_enn.py \
+ --checkpoint work_dirs/csn/finetune_ucf101_csn_edlnokl_avuc_debias/latest.pth \
+ --ind_data ${IND_DATA} \
+ --ood_data ${OOD_DATA} \
+ --uncertainty EDL \
+ --result_prefix ${RESULT_DIR}/CSN_EDLNoKLAvUCDebias_EDL_${OOD_DATASET}
+ ;;
+ *)
+ echo "Invalid model: "${MODEL}
+ exit
+ ;;
+esac
+
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/csn/run_openness.sh b/Downstream/Open-Set-Action-Recognition/experiments/csn/run_openness.sh
new file mode 100644
index 0000000..4630e6a
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/csn/run_openness.sh
@@ -0,0 +1,37 @@
+#!/bin/bash
+
+export CUDA_HOME='/usr/local/cuda'
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+OOD_DATA=$1 # HMDB or MiT
+case ${OOD_DATA} in
+ HMDB)
+ NUM_CLASSES=51
+ ;;
+ MiT)
+ NUM_CLASSES=305
+ ;;
+ *)
+ echo "Invalid OOD Dataset: "${OOD_DATA}
+ exit
+ ;;
+esac
+
+# OOD Detection comparison
+python experiments/compare_openness.py \
+ --base_model csn \
+ --baselines CSN_DNN_BALD CSN_EDLNoKLAvUCDebias_EDL \
+ --thresholds 0.000089 0.004563 \
+ --styles b r \
+ --ood_data ${OOD_DATA} \
+ --ood_ncls ${NUM_CLASSES} \
+ --ind_ncls 101 \
+ --result_png F1_openness_compare_${OOD_DATA}.png
+
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/csn/run_reliability_evaluation.sh b/Downstream/Open-Set-Action-Recognition/experiments/csn/run_reliability_evaluation.sh
new file mode 100644
index 0000000..23745fe
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/csn/run_reliability_evaluation.sh
@@ -0,0 +1,35 @@
+#!/bin/bash
+
+export CUDA_HOME='/usr/local/cuda'
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+OOD_DATASET=$1
+MODEL=$2
+RESULT_DIR='experiments/csn/results'
+
+case ${MODEL} in
+ dnn)
+ # DNN with Dropout model
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/evaluate_calibration.py \
+ --ood_result ${RESULT_DIR}/CSN_DNN_BALD_${OOD_DATASET}_result.npz \
+ --save_prefix ${RESULT_DIR}/../results_reliability/CSN_DNN_BALD_${OOD_DATASET}_reliability
+ ;;
+ edlnokl_avuc_debias)
+ # Evidential Deep Learning (without KL divergence loss term) with AvU Calibration and Debiasing
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/evaluate_calibration.py \
+ --ood_result ${RESULT_DIR}/CSN_EDLNoKLAvUCDebias_EDL_${OOD_DATASET}_result.npz \
+ --save_prefix ${RESULT_DIR}/../results_reliability/CSN_EDLNoKLAvUCDebias_EDL_${OOD_DATASET}_reliability
+ ;;
+ *)
+ echo "Invalid model: "${MODEL}
+ exit
+ ;;
+esac
+
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/demo.py b/Downstream/Open-Set-Action-Recognition/experiments/demo.py
new file mode 100644
index 0000000..56df160
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/demo.py
@@ -0,0 +1,134 @@
+import argparse
+import os
+import torch
+from mmaction.apis import init_recognizer
+from mmcv.parallel import collate, scatter
+from mmaction.datasets.pipelines import Compose
+import numpy as np
+from tqdm import tqdm
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='MMAction2 test')
+ # model config
+ parser.add_argument('config', help='test config file path')
+ parser.add_argument('--ckpt_dear', help='checkpoint file/url')
+ parser.add_argument('--ckpt_nodebias', help='checkpoint file/url')
+ parser.add_argument('--split_file', help='the split file for evaluation')
+ parser.add_argument('--video_path', help='the video path for evaluation')
+ parser.add_argument('--device', type=str, default='cuda:0', help='CPU/CUDA device option')
+ parser.add_argument('--result_list', help='result file prefix')
+ args = parser.parse_args()
+ return args
+
+
+def set_deterministic(seed):
+ torch.manual_seed(seed)
+ torch.cuda.manual_seed(seed)
+ torch.cuda.manual_seed_all(seed) # if you are using multi-GPU.
+ np.random.seed(seed) # Numpy module.
+ torch.backends.cudnn.benchmark = True
+ torch.backends.cudnn.deterministic = True
+
+
+def init_inference(config, checkpoint):
+ # build the recognizer from a config file and checkpoint file/url
+ model = init_recognizer(config, checkpoint, device=device, use_frames=False)
+ cfg = model.cfg
+ cfg.data.test.test_mode = True
+ cfg.data.test.ann_file = args.split_file
+ cfg.data.test.data_prefix = args.video_path
+ evidence = cfg.get('evidence', 'exp')
+ return model, evidence
+
+
+def parse_listfile(list_file, videos_path):
+ assert os.path.exists(list_file), 'split file does not exist! %s'%(list_file)
+ assert os.path.exists(videos_path), 'video path does not exist! %s'%(videos_path)
+ # parse file list
+ filelist, labels = [], []
+ with open(list_file, 'r') as f:
+ for line in f.readlines():
+ videofile = line.strip().split(' ')[0]
+ label = int(line.strip().split(' ')[1])
+ videofile_full = os.path.join(videos_path, videofile)
+ assert os.path.exists(videofile_full), 'video file does not exist! %s'%(videofile_full)
+ filelist.append(videofile_full)
+ labels.append(label)
+ return filelist, labels
+
+
+def run_evidence_inference(model, video_path, evidence='exp'):
+ """Inference a video with the detector.
+
+ Args:
+ model (nn.Module): The loaded recognizer.
+ video_path (str): The video file path/url or the rawframes directory
+ path. If ``use_frames`` is set to True, it should be rawframes
+ directory path. Otherwise, it should be video file path.
+ """
+ cfg = model.cfg
+ device = next(model.parameters()).device # model device
+ # build the data pipeline
+ test_pipeline = cfg.data.test.pipeline
+ test_pipeline = Compose(test_pipeline)
+ # prepare data (by default, we use videodata)
+ start_index = cfg.data.test.get('start_index', 0)
+ data = dict(filename=video_path, label=-1, start_index=start_index, modality='RGB')
+ data = test_pipeline(data)
+ data = collate([data], samples_per_gpu=1)
+ if next(model.parameters()).is_cuda:
+ # scatter to specified GPU
+ data = scatter(data, [device])[0]
+
+ # get the evidence function
+ if evidence == 'relu':
+ from mmaction.models.losses.edl_loss import relu_evidence as get_evidence
+ elif evidence == 'exp':
+ from mmaction.models.losses.edl_loss import exp_evidence as get_evidence
+ elif evidence == 'softplus':
+ from mmaction.models.losses.edl_loss import softplus_evidence as get_evidence
+ else:
+ raise NotImplementedError
+ num_classes = model.cls_head.num_classes
+
+ # forward the model
+ with torch.no_grad():
+ output = model(return_loss=False, **data)[0] # batchsize = 1
+ evidence = get_evidence(torch.from_numpy(output))
+ alpha = evidence + 1
+ uncertainty = num_classes / torch.sum(alpha, dim=0)
+ scores = alpha / torch.sum(alpha, dim=0, keepdim=True)
+ return scores.cpu().numpy(), uncertainty.cpu().numpy()
+
+
+def main():
+ model_dear, evidence_dear = init_inference(args.config, args.ckpt_dear)
+ model_nodebias, evidence_nodebias = init_inference(args.config, args.ckpt_nodebias)
+ # result file
+ result_path = os.path.dirname(args.result_list)
+ if not os.path.exists(result_path):
+ os.makedirs(result_path)
+ fid = open(args.result_list, 'w')
+ # run inference
+ videofiles, labels = parse_listfile(args.split_file, args.video_path)
+ for i, (videofile, label) in tqdm(enumerate(zip(videofiles, labels)), total=len(videofiles)):
+ scores_dear, uncertainty_dear = run_evidence_inference(model_dear, videofile, evidence_dear) # (101,)
+ scores_nodebias, uncertainty_nodebias = run_evidence_inference(model_nodebias, videofile, evidence_nodebias) # (101,)
+ # save
+ pred_dear = int(np.argmax(scores_dear))
+ pred_nodebias = int(np.argmax(scores_nodebias))
+ if pred_dear == label and pred_nodebias != label:
+ fid.writelines('%s %d %d %.6lf %d %.6lf\n'%(videofile, label, pred_dear, float(uncertainty_dear), pred_nodebias, float(uncertainty_nodebias)))
+ fid.close()
+
+
+if __name__ == '__main__':
+
+ args = parse_args()
+ # assign the desired device.
+ device = torch.device(args.device)
+ set_deterministic(1234)
+
+ main()
+
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/draw_confusion_matrix.py b/Downstream/Open-Set-Action-Recognition/experiments/draw_confusion_matrix.py
new file mode 100644
index 0000000..c0fb1af
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/draw_confusion_matrix.py
@@ -0,0 +1,209 @@
+import numpy as np
+import argparse
+import os
+import matplotlib.pyplot as plt
+# from mmaction.core.evaluation import confusion_matrix
+from mpl_toolkits.axes_grid1 import make_axes_locatable
+import matplotlib.ticker as ticker
+from matplotlib.colors import LogNorm
+
+
+def confusion_maxtrix(ind_labels, ind_results, ind_uncertainties,
+ ood_labels, ood_results, ood_uncertainties,
+ threshold, know_ood_labels=False, normalize=True):
+ num_indcls = max(ind_labels) + 1
+ num_oodcls = max(ood_labels) + 1
+ confmat = np.zeros((num_indcls + num_oodcls, num_indcls + num_oodcls), dtype=np.float32)
+ for rlabel, plabel, uncertain in zip(ind_labels, ind_results, ind_uncertainties):
+ if uncertain > threshold:
+ # known --> unknown (bottom-left)
+ confmat[num_indcls:num_indcls+num_oodcls, rlabel] += 1.0 * num_oodcls
+ else:
+ # known --> known (top-left)
+ confmat[plabel, rlabel] += 1.0 * num_indcls
+ if know_ood_labels:
+ for rlable, plabel, uncertain in zip(ood_labels, ood_results, ood_uncertainties):
+ if uncertain > threshold:
+ # unknown --> unknown (bottom-right)
+ confmat[num_indcls:num_indcls+num_oodcls, num_indcls+rlable] += 1.0
+ else:
+ # unknown --> known (top-right)
+ confmat[plabel, num_indcls+rlable] += 1.0 * num_oodcls
+ else:
+ for plabel, uncertain in zip(ood_results, ood_uncertainties):
+ if uncertain > threshold:
+ # unknown --> unknown (bottom-right)
+ confmat[num_indcls:num_indcls+num_oodcls, num_indcls:num_indcls+num_oodcls] += 1.0
+ else:
+ # unknown --> known (top-right)
+ confmat[plabel, num_indcls:num_indcls+num_oodcls] += 1 * num_oodcls
+ if normalize:
+ minval = np.min(confmat[np.nonzero(confmat)])
+ maxval = np.max(confmat)
+ confmat = (confmat - minval) / (maxval - minval + 1e-6)
+ # confmat = np.nan_to_num(confmat)
+ return confmat
+
+
+
+def confusion_maxtrix_top(ind_labels, ind_results, ind_uncertainties,
+ ood_labels, ood_results, ood_uncertainties,
+ threshold, normalize=True):
+ num_indcls = max(ind_labels) + 1
+ num_oodcls = max(ood_labels) + 1
+ confmat = np.ones((num_indcls, num_indcls + num_oodcls), dtype=np.float32) # (K, K+C) white
+ # process in-distribution results
+ for rlabel, plabel, uncertain in zip(ind_labels, ind_results, ind_uncertainties):
+ if uncertain < threshold:
+ # known --> known (top-left)
+ confmat[plabel, rlabel] -= 1.0 / len(ind_results) # make it darker
+ # process out-of-distribution results
+ for rlable, plabel, uncertain in zip(ood_labels, ood_results, ood_uncertainties):
+ if uncertain < threshold:
+ # unknown --> known (top-right)
+ confmat[plabel, num_indcls+rlable] -= 1.0 / len(ood_results) # make it darker
+ if normalize:
+ minval = np.min(confmat)
+ maxval = np.max(confmat)
+ confmat = (confmat - minval) / (maxval - minval + 1e-6) # normalize to [0, 1]
+ return confmat
+
+
+
+def plot_confmat(confmat, know_ood_labels=False):
+ fig = plt.figure(figsize=(4,4))
+ plt.rcParams["font.family"] = "Arial" # Times New Roman
+ fontsize = 20
+ ax = plt.gca()
+ confmat_vis = confmat.copy()
+ im = ax.imshow(confmat_vis, cmap='hot')
+ plt.xticks(fontsize=fontsize)
+ plt.yticks(fontsize=fontsize)
+ divider = make_axes_locatable(ax)
+ cax = divider.append_axes("right", size="5%", pad=0.05)
+ cbar = plt.colorbar(im, cax=cax)
+ # cbar.locator = ticker.MaxNLocator(nbins=5)
+ # # barticks = np.linspace(np.min(confmat) * 1000, np.max(confmat) * 1000, 5).tolist()
+ # # cbar.set_ticks(barticks)
+ # cbar.ax.tick_params(labelsize=fontsize)
+ cbar.set_ticks([])
+ cbar.update_ticks()
+ plt.tight_layout()
+ save_file = args.save_file[:-4] + '_knownOOD.png' if know_ood_labels else args.save_file
+ plt.savefig(save_file, bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ plt.savefig(save_file[:-4] + '.pdf', bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ plt.close()
+
+def get_topk_2d(arr, topk=5):
+ col_indices = np.argmax(arr, axis=1) # column indices
+ vals = [arr[r, c] for r, c in enumerate(col_indices)]
+ row_indices = np.argsort(vals)[::-1] # decreasing sort
+ result_inds = np.zeros((topk, 2), dtype=np.int32)
+ result_vals = []
+ for k in range(topk):
+ result_inds[k, 0] = row_indices[k]
+ result_inds[k, 1] = col_indices[row_indices[k]]
+ result_vals.append(vals[row_indices[k]])
+ return result_inds, result_vals
+
+def read_classnames(list_file):
+ names = []
+ with open(list_file, 'r') as f:
+ for line in f.readlines():
+ names.append(line.strip().split(' ')[-1])
+ return names
+
+def plot_top_confmat(confmat):
+ ind_mappings = 'data/ucf101/annotations/classInd.txt'
+ ind_cls_names = read_classnames(ind_mappings)
+ ood_mappings = 'data/hmdb51/annotations/classInd.txt'
+ ood_cls_names = read_classnames(ood_mappings)
+
+ fig = plt.figure(figsize=(8,4))
+ plt.rcParams["font.family"] = "Arial" # Times New Roman
+ fontsize = 15
+ ax = plt.gca()
+ confmat_vis = confmat.copy()
+ im = ax.imshow(confmat_vis, cmap='hot')
+ plt.axvline(num_indcls-1, 0, num_indcls-1, linestyle='--')
+ # find the top-K mis-classification for unknown
+ result_inds, result_vals = get_topk_2d(1 - confmat_vis[:, num_indcls+1:])
+ ood_ids = np.argmin(confmat_vis[:, num_indcls+1:], axis=1)
+ text_offset = [[-25, 1], [-25, -3], [-61, 1], [-28, 2], [-32, 1]]
+ for i, (r, c) in enumerate(result_inds):
+ hd = plt.Circle((c + num_indcls, r), 5, fill=False)
+ ax.set_aspect(1)
+ ax.add_artist(hd )
+ off_c, off_r = 6, 1
+ if i == 1:
+ off_c, off_r = -4, -6
+ plt.text(c + num_indcls + off_c, r + off_r, ood_cls_names[c], color='blue', fontsize=fontsize)
+ plt.plot([num_indcls, num_indcls + c], [r, r], 'r--')
+ plt.text(num_indcls + text_offset[i][0], r+text_offset[i][1], ind_cls_names[r], color='red', fontsize=fontsize)
+ plt.ylabel('Predicted Classes', fontsize=fontsize)
+ plt.xlabel('UCF-101 (known) + HMDB-51 (unknown)', fontsize=fontsize)
+ plt.xticks(fontsize=fontsize)
+ plt.yticks(fontsize=fontsize)
+ plt.tight_layout()
+ plt.savefig(args.save_file[:-4] + '_top.png', bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ plt.savefig(args.save_file[:-4] + '_top.pdf', bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ plt.close()
+
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser(description='MMAction2 test')
+ # model config
+ parser.add_argument('--ood_result', help='the result file of ood detection')
+ parser.add_argument('--uncertain_thresh', type=float, default=0.0001, help='the threshold value for prediction')
+ parser.add_argument('--top_part', action='store_true', help='Whether to show the top part of confmat separately.')
+ parser.add_argument('--save_file', help='the image file path of generated confusion matrix')
+ args = parser.parse_args()
+
+ results = np.load(args.ood_result, allow_pickle=True)
+ ind_uncertainties = results['ind_unctt'] # (N1,)
+ ood_uncertainties = results['ood_unctt'] # (N2,)
+ ind_results = results['ind_pred'] # (N1,)
+ ood_results = results['ood_pred'] # (N2,)
+ ind_labels = results['ind_label']
+ ood_labels = results['ood_label']
+
+ # result path
+ result_path = os.path.dirname(args.save_file)
+ if not os.path.exists(result_path):
+ os.makedirs(result_path)
+
+ # OOD classes are unknown
+ confmat1 = confusion_maxtrix(ind_labels, ind_results, ind_uncertainties,
+ ood_labels, ood_results, ood_uncertainties,
+ args.uncertain_thresh, know_ood_labels=False)
+ plot_confmat(confmat1, know_ood_labels=False)
+
+ num_indcls = max(ind_labels) + 1
+ num_oodcls = max(ood_labels) + 1
+ UKC_value = np.mean(confmat1[:num_indcls, num_indcls:]) # unknown --> known (top-right)
+ UUC_value = np.mean(confmat1[num_indcls:, num_indcls:]) # unknown --> unknown (bottom-right)
+ KUC_value = np.mean(confmat1[num_indcls:, :num_indcls]) # known --> unknown (bottom-left)
+ KKC_value = np.mean(np.diag(confmat1[:num_indcls, :num_indcls])) # known --> known (top-left)
+ print("The average UUC=: %.6lf, UKC=%.6lf, KUC=%.6lf, KKC=%.6lf"%(UUC_value, UKC_value, KUC_value, KKC_value))
+
+
+
+ # # OOD classes are known
+ # confmat2 = confusion_maxtrix(ind_labels, ind_results, ind_uncertainties,
+ # ood_labels, ood_results, ood_uncertainties,
+ # args.uncertain_thresh, know_ood_labels=True)
+ # plot_confmat(confmat2, know_ood_labels=True)
+
+ # # save the confusion matrix for further analysis
+ # np.savez(args.save_file[:-4], confmat_unknown_ood=confmat1, confmat_known_ood=confmat2)
+ # votes_ind = np.sum(confmat1[:101, 101:], axis=1)
+ # print("Top-10 false positive IND classes: ", np.argsort(votes_ind)[-10:])
+
+ # votes_ood = np.sum(confmat1[101:, :101], axis=1)
+ # print("Top-10 false negative IND classes: ", np.argsort(votes_ood)[-10:])
+
+ if args.top_part:
+ top_confmat = confusion_maxtrix_top(ind_labels, ind_results, ind_uncertainties,
+ ood_labels, ood_results, ood_uncertainties,
+ args.uncertain_thresh)
+ plot_top_confmat(top_confmat)
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/draw_fig7cd.py b/Downstream/Open-Set-Action-Recognition/experiments/draw_fig7cd.py
new file mode 100644
index 0000000..b76f343
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/draw_fig7cd.py
@@ -0,0 +1,70 @@
+import os
+import numpy as np
+import matplotlib.pyplot as plt
+
+
+def plot_by_uncertainty(result_file, uncertainty='EDL', auc=80, fontsize=16, result_prefix=''):
+ assert os.path.exists(result_file), 'result file not exists! %s'%(result_file)
+ results = np.load(result_file, allow_pickle=True)
+ # ind_confidences = results['ind_conf']
+ # ood_confidences = results['ood_conf']
+ ind_uncertainties = results['ind_unctt'] # (N1,)
+ ood_uncertainties = results['ood_unctt'] # (N2,)
+ ind_results = results['ind_pred'] # (N1,)
+ ood_results = results['ood_pred'] # (N2,)
+ ind_labels = results['ind_label']
+ ood_labels = results['ood_label']
+
+ # visualize
+ ind_uncertainties = np.array(ind_uncertainties)
+ ind_uncertainties = (ind_uncertainties-np.min(ind_uncertainties)) / (np.max(ind_uncertainties) - np.min(ind_uncertainties)) # normalize
+ ood_uncertainties = np.array(ood_uncertainties)
+ ood_uncertainties = (ood_uncertainties-np.min(ood_uncertainties)) / (np.max(ood_uncertainties) - np.min(ood_uncertainties)) # normalize
+
+ fig = plt.figure(figsize=(5,4)) # (w, h)
+ plt.rcParams["font.family"] = "Arial" # Times New Roman
+ data_label = 'HMDB-51' if ood_data == 'HMDB' else 'MiT-v2'
+ counts, bins, bars = plt.hist([ind_uncertainties, ood_uncertainties], 50,
+ density=True, histtype='bar', color=['blue', 'red'],
+ label=['in-distribution (%s)'%(ind_data), 'out-of-distribution (%s)'%(data_label)])
+ plt.legend(fontsize=fontsize-3)
+ plt.text(0.6, 6, 'AUC = %.2lf'%(auc), fontsize=fontsize-3)
+ plt.xlabel('%s uncertainty'%(uncertainty), fontsize=fontsize)
+ plt.ylabel('Density', fontsize=fontsize)
+ plt.xticks(fontsize=fontsize)
+ plt.yticks(fontsize=fontsize)
+ plt.xlim(0, 1.01)
+ plt.ylim(0, 10.01)
+ plt.tight_layout()
+
+ result_dir = os.path.dirname(result_prefix)
+ if not os.path.exists(result_dir):
+ os.makedirs(result_dir)
+ # save the figure
+ plt.savefig(os.path.join(result_prefix + '_distribution.png'), bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ plt.savefig(os.path.join(result_prefix + '_distribution.pdf'), bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ return counts, bins, bars
+
+
+if __name__ == '__main__':
+
+ fontsize = 20
+ ind_data = 'UCF-101'
+ ood_data = 'MiT'
+ # DRIVE (vanilla)
+ result_file = 'i3d/results/I3D_EDLNoKL_EDL_%s_result.npz'%(ood_data)
+ counts, bins, bars = plot_by_uncertainty(result_file, uncertainty='EDL', auc=81.43, fontsize=fontsize, result_prefix='temp_rebuttal/I3D_MiT_Vanilla')
+ counts = counts[:, 3:]
+ bins = bins[3:]
+ # mode = np.argsort(counts[1, :])[:5]
+ mode = np.argmax(counts[1, :])
+ print('the most frequent bin:(' + str(bins[mode]) + ',' + str(bins[mode+1]) + ')')
+
+ # DRIVE (full)
+ result_file = 'i3d/results/I3D_EDLNoKLAvUCCED_EDL_%s_result.npz'%(ood_data)
+ counts, bins, bars = plot_by_uncertainty(result_file, uncertainty='EDL', auc=81.54, fontsize=fontsize, result_prefix='temp_rebuttal/I3D_MiT_Full')
+ counts = counts[:, 3:]
+ bins = bins[3:]
+ # mode = np.argsort(counts[1, :])[:5]
+ mode = np.argmax(counts[1, :])
+ print('the most frequent bin:(' + str(bins[mode]) + ',' + str(bins[mode+1]) + ')')
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/draw_ood_hist.py b/Downstream/Open-Set-Action-Recognition/experiments/draw_ood_hist.py
new file mode 100644
index 0000000..2d6f7dd
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/draw_ood_hist.py
@@ -0,0 +1,306 @@
+import argparse
+import os
+import numpy as np
+import matplotlib.pyplot as plt
+import matplotlib.ticker as ticker
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='Draw histogram')
+ parser.add_argument('--uncertainty', default='EDL', choices=['BALD', 'Entropy', 'EDL'], help='the uncertainty estimation method')
+ parser.add_argument('--ind_data', default='UCF-101', help='the split file of in-distribution testing data')
+ parser.add_argument('--ood_data', default='HMDB', choices=['HMDB', 'MiT'], help='the split file of out-of-distribution testing data')
+ parser.add_argument('--model', default='I3D', choices=['I3D', 'TSM', 'SlowFast', 'TPN'], help='the action recognition model.')
+ parser.add_argument('--result_prefix', default='temp/temp.png', help='result file prefix')
+ args = parser.parse_args()
+ return args
+
+def plot_by_uncertainty(result_file, uncertainty='EDL', auc=80, fontsize=16):
+ assert os.path.exists(result_file), 'result file not exists! %s'%(result_file)
+ results = np.load(result_file, allow_pickle=True)
+ # ind_confidences = results['ind_conf']
+ # ood_confidences = results['ood_conf']
+ ind_uncertainties = results['ind_unctt'] # (N1,)
+ ood_uncertainties = results['ood_unctt'] # (N2,)
+ ind_results = results['ind_pred'] # (N1,)
+ ood_results = results['ood_pred'] # (N2,)
+ ind_labels = results['ind_label']
+ ood_labels = results['ood_label']
+
+ # visualize
+ ind_uncertainties = np.array(ind_uncertainties)
+ ind_uncertainties = (ind_uncertainties-np.min(ind_uncertainties)) / (np.max(ind_uncertainties) - np.min(ind_uncertainties)) # normalize
+ ood_uncertainties = np.array(ood_uncertainties)
+ ood_uncertainties = (ood_uncertainties-np.min(ood_uncertainties)) / (np.max(ood_uncertainties) - np.min(ood_uncertainties)) # normalize
+
+ fig = plt.figure(figsize=(5,4)) # (w, h)
+ plt.rcParams["font.family"] = "Arial" # Times New Roman
+ data_label = 'HMDB-51' if args.ood_data == 'HMDB' else 'MiT-v2'
+ plt.hist([ind_uncertainties, ood_uncertainties], 50,
+ density=True, histtype='bar', color=['blue', 'red'],
+ label=['in-distribution (%s)'%(args.ind_data), 'out-of-distribution (%s)'%(data_label)])
+ plt.legend(fontsize=fontsize-3)
+ plt.text(0.6, 6, 'AUC = %.2lf'%(auc), fontsize=fontsize-3)
+ plt.xlabel('%s uncertainty'%(uncertainty), fontsize=fontsize)
+ plt.ylabel('Density', fontsize=fontsize)
+ plt.xticks(fontsize=fontsize)
+ plt.yticks(fontsize=fontsize)
+ plt.xlim(0, 1.01)
+ plt.ylim(0, 10.01)
+ plt.tight_layout()
+
+ result_dir = os.path.dirname(args.result_prefix)
+ if not os.path.exists(result_dir):
+ os.makedirs(result_dir)
+ # save the figure
+ plt.savefig(os.path.join(args.result_prefix + '_distribution.png'), bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ plt.savefig(os.path.join(args.result_prefix + '_distribution.pdf'), bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+
+
+def get_confidence(result_file, conf='softmax'):
+ # only for SoftMax and OpenMax
+ assert os.path.exists(result_file), 'result file not exists! %s'%(result_file)
+ results = np.load(result_file, allow_pickle=True)
+ if conf == 'softmax':
+ ind_score = results['ind_softmax'] # (N1, C)
+ ood_score = results['ood_softmax'] # (N2, C)
+ else:
+ ind_score = results['ind_openmax'] # (N1, C+1)
+ ood_score = results['ood_openmax'] # (N2, C+1)
+ ind_conf = np.max(ind_score, axis=1)
+ ood_conf = np.max(ood_score, axis=1)
+ return ind_conf, ood_conf
+
+
+def plot_by_confidence(ind_confidence, ood_confidence, auc=80, fontsize=16):
+ # visualize
+ ind_conf = ind_confidence.copy()
+ ind_conf = (ind_conf-np.min(ind_conf)) / (np.max(ind_conf) - np.min(ind_conf) + 1e-6) # normalize
+ ood_conf = ood_confidence.copy()
+ ood_conf = (ood_conf-np.min(ood_conf)) / (np.max(ood_conf) - np.min(ood_conf) + 1e-6) # normalize
+
+ fig = plt.figure(figsize=(5,4)) # (w, h)
+ plt.rcParams["font.family"] = "Arial" # Times New Roman
+ data_label = 'HMDB-51' if args.ood_data == 'HMDB' else 'MiT-v2'
+ plt.hist([ind_conf, ood_conf], 50,
+ density=True, histtype='bar', color=['blue', 'red'],
+ label=['in-distribution (%s)'%(args.ind_data), 'out-of-distribution (%s)'%(data_label)])
+ plt.legend(fontsize=fontsize-3)
+ plt.text(0.6, 6, 'AUC = %.2lf'%(auc), fontsize=fontsize-3)
+ plt.xlabel('Confidence', fontsize=fontsize)
+ plt.ylabel('Density', fontsize=fontsize)
+ plt.xticks(fontsize=fontsize)
+ plt.yticks(fontsize=fontsize)
+ plt.xlim(0, 1.01)
+ plt.ylim(0, 10.01)
+ plt.tight_layout()
+
+ result_dir = os.path.dirname(args.result_prefix)
+ if not os.path.exists(result_dir):
+ os.makedirs(result_dir)
+ # save the figure
+ plt.savefig(os.path.join(args.result_prefix + '_distribution.png'), bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ plt.savefig(os.path.join(args.result_prefix + '_distribution.pdf'), bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+
+def main_i3d():
+ # common settings
+ fontsize = 18 if args.ood_data == 'HMDB' else 20
+
+ # SoftMax
+ result_file = 'i3d/results_baselines/openmax/I3D_OpenMax_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'i3d/results_baselines/softmax/I3D_SoftMax_Conf_%s'%(args.ood_data)
+ auc = 75.68 if args.ood_data == 'HMDB' else 79.94
+ ind_conf, ood_conf = get_confidence(result_file, conf='softmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # OpenMax
+ result_file = 'i3d/results_baselines/openmax/I3D_OpenMax_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'i3d/results_baselines/openmax/I3D_OpenMax_Conf_%s'%(args.ood_data)
+ auc = 74.34 if args.ood_data == 'HMDB' else 77.76
+ ind_conf, ood_conf = get_confidence(result_file, conf='openmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # RPL
+ result_file = 'i3d/results_baselines/rpl/I3D_RPL_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'i3d/results_baselines/rpl/I3D_RPL_Conf_%s'%(args.ood_data)
+ auc = 75.20 if args.ood_data == 'HMDB' else 79.16
+ ind_conf, ood_conf = get_confidence(result_file, conf='softmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # MC Dropout
+ result_file = 'i3d/results/I3D_DNN_BALD_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'i3d/results/I3D_DNN_BALD_%s'%(args.ood_data)
+ auc = 75.07 if args.ood_data == 'HMDB' else 79.14
+ plot_by_uncertainty(result_file, uncertainty='BALD', auc=auc, fontsize=fontsize)
+
+ # BNN SVI
+ result_file = 'i3d/results/I3D_BNN_BALD_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'i3d/results/I3D_BNN_BALD_%s'%(args.ood_data)
+ auc = 74.66 if args.ood_data == 'HMDB' else 79.50
+ plot_by_uncertainty(result_file, uncertainty='BALD', auc=auc, fontsize=fontsize)
+
+ # DRIVE (vanilla)
+ result_file = 'i3d/results/I3D_EDLNoKL_EDL_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'i3d/results/I3D_EDLNoKL_EDL_%s'%(args.ood_data)
+ auc = 76.41 if args.ood_data == 'HMDB' else 81.43
+ plot_by_uncertainty(result_file, uncertainty='EDL', auc=auc, fontsize=fontsize)
+
+ # DRIVE (full)
+ result_file = 'i3d/results/I3D_EDLNoKLAvUCCED_EDL_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'i3d/results/I3D_EDLNoKLAvUCCED_EDL_%s'%(args.ood_data)
+ auc = 77.08 if args.ood_data == 'HMDB' else 81.54
+ plot_by_uncertainty(result_file, uncertainty='EDL', auc=auc, fontsize=fontsize)
+
+
+def main_tsm():
+ # common settings
+ fontsize = 18 if args.ood_data == 'HMDB' else 20
+
+ # SoftMax
+ result_file = 'tsm/results_baselines/openmax/TSM_OpenMax_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tsm/results_baselines/softmax/TSM_SoftMax_Conf_%s'%(args.ood_data)
+ auc = 77.99 if args.ood_data == 'HMDB' else 82.38
+ ind_conf, ood_conf = get_confidence(result_file, conf='softmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # OpenMax
+ result_file = 'tsm/results_baselines/openmax/TSM_OpenMax_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tsm/results_baselines/openmax/TSM_OpenMax_Conf_%s'%(args.ood_data)
+ auc = 77.07 if args.ood_data == 'HMDB' else 83.05
+ ind_conf, ood_conf = get_confidence(result_file, conf='openmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # RPL
+ result_file = 'tsm/results_baselines/rpl/TSM_RPL_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tsm/results_baselines/rpl/TSM_RPL_Conf_%s'%(args.ood_data)
+ auc = 73.62 if args.ood_data == 'HMDB' else 77.28
+ ind_conf, ood_conf = get_confidence(result_file, conf='softmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # MC Dropout
+ result_file = 'tsm/results/TSM_DNN_BALD_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tsm/results/TSM_DNN_BALD_%s'%(args.ood_data)
+ auc = 73.85 if args.ood_data == 'HMDB' else 78.35
+ plot_by_uncertainty(result_file, uncertainty='BALD', auc=auc, fontsize=fontsize)
+
+ # BNN SVI
+ result_file = 'tsm/results/TSM_BNN_BALD_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tsm/results/TSM_BNN_BALD_%s'%(args.ood_data)
+ auc = 73.42 if args.ood_data == 'HMDB' else 77.39
+ plot_by_uncertainty(result_file, uncertainty='BALD', auc=auc, fontsize=fontsize)
+
+ # DRIVE (full)
+ result_file = 'tsm/results/TSM_EDLNoKLAvUCDebias_EDL_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tsm/results/TSM_EDLNoKLAvUCDebias_EDL_%s'%(args.ood_data)
+ auc = 78.65 if args.ood_data == 'HMDB' else 83.92
+ plot_by_uncertainty(result_file, uncertainty='EDL', auc=auc, fontsize=fontsize)
+
+
+def main_slowfast():
+ # common settings
+ fontsize = 18 if args.ood_data == 'HMDB' else 20
+
+ # SoftMax
+ result_file = 'slowfast/results_baselines/openmax/SlowFast_OpenMax_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'slowfast/results_baselines/softmax/SlowFast_SoftMax_Conf_%s'%(args.ood_data)
+ auc = 79.16 if args.ood_data == 'HMDB' else 82.88
+ ind_conf, ood_conf = get_confidence(result_file, conf='softmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # OpenMax
+ result_file = 'slowfast/results_baselines/openmax/SlowFast_OpenMax_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'slowfast/results_baselines/openmax/SlowFast_OpenMax_Conf_%s'%(args.ood_data)
+ auc = 78.76 if args.ood_data == 'HMDB' else 80.62
+ ind_conf, ood_conf = get_confidence(result_file, conf='openmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # RPL
+ result_file = 'slowfast/results_baselines/rpl/SlowFast_RPL_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'slowfast/results_baselines/rpl/SlowFast_RPL_Conf_%s'%(args.ood_data)
+ auc = 74.23 if args.ood_data == 'HMDB' else 77.42
+ ind_conf, ood_conf = get_confidence(result_file, conf='softmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # MC Dropout
+ result_file = 'slowfast/results/SlowFast_DNN_BALD_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'slowfast/results/SlowFast_DNN_BALD_%s'%(args.ood_data)
+ auc = 75.41 if args.ood_data == 'HMDB' else 78.49
+ plot_by_uncertainty(result_file, uncertainty='BALD', auc=auc, fontsize=fontsize)
+
+ # BNN SVI
+ result_file = 'slowfast/results/SlowFast_BNN_BALD_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'slowfast/results/SlowFast_BNN_BALD_%s'%(args.ood_data)
+ auc = 74.78 if args.ood_data == 'HMDB' else 77.39
+ plot_by_uncertainty(result_file, uncertainty='BALD', auc=auc, fontsize=fontsize)
+
+ # DRIVE (full)
+ result_file = 'slowfast/results/SlowFast_EDLNoKLAvUCDebias_EDL_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'slowfast/results/SlowFast_EDLNoKLAvUCDebias_EDL_%s'%(args.ood_data)
+ auc = 82.94 if args.ood_data == 'HMDB' else 86.99
+ plot_by_uncertainty(result_file, uncertainty='EDL', auc=auc, fontsize=fontsize)
+
+
+def main_tpn():
+ # common settings
+ fontsize = 18 if args.ood_data == 'HMDB' else 20
+
+ # SoftMax
+ result_file = 'tpn_slowonly/results_baselines/openmax/TPN_OpenMax_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tpn_slowonly/results_baselines/softmax/TPN_SoftMax_Conf_%s'%(args.ood_data)
+ auc = 77.97 if args.ood_data == 'HMDB' else 81.35
+ ind_conf, ood_conf = get_confidence(result_file, conf='softmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # OpenMax
+ result_file = 'tpn_slowonly/results_baselines/openmax/TPN_OpenMax_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tpn_slowonly/results_baselines/openmax/TPN_OpenMax_Conf_%s'%(args.ood_data)
+ auc = 74.12 if args.ood_data == 'HMDB' else 76.26
+ ind_conf, ood_conf = get_confidence(result_file, conf='openmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # RPL
+ result_file = 'tpn_slowonly/results_baselines/rpl/TPN_RPL_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tpn_slowonly/results_baselines/rpl/TPN_RPL_Conf_%s'%(args.ood_data)
+ auc = 75.32 if args.ood_data == 'HMDB' else 78.21
+ ind_conf, ood_conf = get_confidence(result_file, conf='softmax')
+ plot_by_confidence(ind_conf, ood_conf, auc=auc, fontsize=fontsize)
+
+ # MC Dropout
+ result_file = 'tpn_slowonly/results/TPN_SlowOnly_Dropout_BALD_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tpn_slowonly/results/TPN_SlowOnly_Dropout_BALD_%s'%(args.ood_data)
+ auc = 74.13 if args.ood_data == 'HMDB' else 77.76
+ plot_by_uncertainty(result_file, uncertainty='BALD', auc=auc, fontsize=fontsize)
+
+ # BNN SVI
+ result_file = 'tpn_slowonly/results/TPN_SlowOnly_BNN_BALD_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tpn_slowonly/results/TPN_SlowOnly_BNN_BALD_%s'%(args.ood_data)
+ auc = 72.68 if args.ood_data == 'HMDB' else 75.32
+ plot_by_uncertainty(result_file, uncertainty='BALD', auc=auc, fontsize=fontsize)
+
+ # DRIVE (full)
+ result_file = 'tpn_slowonly/results/TPN_SlowOnly_EDLlogNoKLAvUCDebias_EDL_%s_result.npz'%(args.ood_data)
+ args.result_prefix = 'tpn_slowonly/results/TPN_SlowOnly_EDLlogNoKLAvUCDebias_EDL_%s'%(args.ood_data)
+ auc = 79.23 if args.ood_data == 'HMDB' else 81.80
+ plot_by_uncertainty(result_file, uncertainty='EDL', auc=auc, fontsize=fontsize)
+
+
+if __name__ == '__main__':
+
+ args = parse_args()
+
+ if args.model == 'I3D':
+ # draw results on I3D
+ main_i3d()
+ elif args.model == 'TSM':
+ # draw results on TSM
+ main_tsm()
+ elif args.model == 'SlowFast':
+ # draw results on SlowFast
+ main_slowfast()
+ elif args.model == 'TPN':
+ # draw results on TPN
+ main_tpn()
+ else:
+ raise NotImplementedError
+
+
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/draw_openness_curves.py b/Downstream/Open-Set-Action-Recognition/experiments/draw_openness_curves.py
new file mode 100644
index 0000000..0b7dcca
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/draw_openness_curves.py
@@ -0,0 +1,323 @@
+import os, argparse
+import numpy as np
+import matplotlib.pyplot as plt
+from sklearn.metrics import f1_score
+
+
+def softmax_curvepoints(result_file, thresh, ood_ncls, num_rand):
+ assert os.path.exists(result_file), "File not found! Run baseline_i3d_softmax.py first!"
+ # load the testing results
+ results = np.load(result_file, allow_pickle=True)
+ ind_softmax = results['ind_softmax'] # (N1, C)
+ ood_softmax = results['ood_softmax'] # (N2, C)
+ ind_labels = results['ind_label'] # (N1,)
+ ood_labels = results['ood_label'] # (N2,)
+
+ ind_ncls = ind_softmax.shape[1]
+ ind_results = np.argmax(ind_softmax, axis=1)
+ ood_results = np.argmax(ood_softmax, axis=1)
+ ind_conf = np.max(ind_softmax, axis=1)
+ ood_conf = np.max(ood_softmax, axis=1)
+
+ ind_results[ind_conf < thresh] = ind_ncls # incorrect rejection
+ # open set F1 score (multi-class)
+ macro_F1 = f1_score(ind_labels, ind_results, average='macro')
+ macro_F1_list = [macro_F1 * 100]
+ openness_list = [0]
+ for n in range(ood_ncls):
+ ncls_novel = n + 1
+ openness = (1 - np.sqrt((2 * ind_ncls) / (2 * ind_ncls + ncls_novel))) * 100
+ openness_list.append(openness)
+ # randoml select the subset of ood samples
+ macro_F1_multi = np.zeros((num_rand), dtype=np.float32)
+ for m in range(num_rand):
+ cls_select = np.random.choice(ood_ncls, ncls_novel, replace=False)
+ ood_sub_results = np.concatenate([ood_results[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_labels = np.ones_like(ood_sub_results) * ind_ncls
+ ood_sub_confs = np.concatenate([ood_conf[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_results[ood_sub_confs < thresh] = ind_ncls # correct rejection
+ # construct preds and labels
+ preds = np.concatenate((ind_results, ood_sub_results), axis=0)
+ labels = np.concatenate((ind_labels, ood_sub_labels), axis=0)
+ macro_F1_multi[m] = f1_score(labels, preds, average='macro')
+ macro_F1 = np.mean(macro_F1_multi) * 100
+ macro_F1_list.append(macro_F1)
+ return openness_list, macro_F1_list
+
+def openmax_curvepoints(result_file, ood_ncls, num_rand):
+ assert os.path.exists(result_file), "File not found! Run baseline_i3d_openmax.py first!"
+ results = np.load(result_file, allow_pickle=True)
+ ind_openmax = results['ind_openmax'] # (N1, C+1)
+ ood_openmax = results['ood_openmax'] # (N2, C+1)
+ ind_labels = results['ind_label'] # (N1,)
+ ood_labels = results['ood_label'] # (N2,)
+ ind_results = np.argmax(ind_openmax, axis=1)
+ ood_results = np.argmax(ood_openmax, axis=1)
+ ind_ncls = ind_openmax.shape[1] - 1 # (C+1)-1
+
+ # open set F1 score (multi-class)
+ macro_F1 = f1_score(ind_labels, ind_results, average='macro')
+ macro_F1_list = [macro_F1 * 100]
+ openness_list = [0]
+ for n in range(ood_ncls):
+ ncls_novel = n + 1
+ openness = (1 - np.sqrt((2 * ind_ncls) / (2 * ind_ncls + ncls_novel))) * 100
+ openness_list.append(openness)
+ # randoml select the subset of ood samples
+ macro_F1_multi = np.zeros((num_rand), dtype=np.float32)
+ for m in range(num_rand):
+ cls_select = np.random.choice(ood_ncls, ncls_novel, replace=False)
+ ood_sub_results = np.concatenate([ood_results[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_labels = np.ones_like(ood_sub_results) * ind_ncls
+ # construct preds and labels
+ preds = np.concatenate((ind_results, ood_sub_results), axis=0)
+ labels = np.concatenate((ind_labels, ood_sub_labels), axis=0)
+ macro_F1_multi[m] = f1_score(labels, preds, average='macro')
+ macro_F1 = np.mean(macro_F1_multi) * 100
+ macro_F1_list.append(macro_F1)
+ return openness_list, macro_F1_list
+
+
+def uncertainty_curvepoints(result_file, thresh, ind_ncls, ood_ncls, num_rand):
+ assert os.path.exists(result_file), "File not found! Run ood_detection first!"
+ # load the testing results
+ results = np.load(result_file, allow_pickle=True)
+ ind_uncertainties = results['ind_unctt'] # (N1,)
+ ood_uncertainties = results['ood_unctt'] # (N2,)
+ ind_results = results['ind_pred'] # (N1,)
+ ood_results = results['ood_pred'] # (N2,)
+ ind_labels = results['ind_label']
+ ood_labels = results['ood_label']
+ # open set F1 score (multi-class)
+ ind_results[ind_uncertainties > thresh] = ind_ncls # falsely rejection
+ macro_F1 = f1_score(ind_labels, ind_results, average='macro')
+ macro_F1_list = [macro_F1 * 100]
+ openness_list = [0]
+ for n in range(ood_ncls):
+ ncls_novel = n + 1
+ openness = (1 - np.sqrt((2 * ind_ncls) / (2 * ind_ncls + ncls_novel))) * 100
+ openness_list.append(openness)
+ # randoml select the subset of ood samples
+ macro_F1_multi = np.zeros((num_rand), dtype=np.float32)
+ for m in range(num_rand):
+ cls_select = np.random.choice(ood_ncls, ncls_novel, replace=False)
+ ood_sub_results = np.concatenate([ood_results[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_uncertainties = np.concatenate([ood_uncertainties[ood_labels == clsid] for clsid in cls_select])
+ ood_sub_results[ood_sub_uncertainties > thresh] = ind_ncls # correctly rejection
+ ood_sub_labels = np.ones_like(ood_sub_results) * ind_ncls
+ # construct preds and labels
+ preds = np.concatenate((ind_results, ood_sub_results), axis=0)
+ labels = np.concatenate((ind_labels, ood_sub_labels), axis=0)
+ macro_F1_multi[m] = f1_score(labels, preds, average='macro')
+ macro_F1 = np.mean(macro_F1_multi) * 100
+ macro_F1_list.append(macro_F1)
+ return openness_list, macro_F1_list
+
+
+def plot_all_curves(openness, values, line_styles, result_prefix, ylim=[60, 80], fontsize=18):
+ fig = plt.figure(figsize=(8,6)) # (w, h)
+ plt.rcParams["font.family"] = "Arial"
+ for k, v in values.items():
+ plt.plot(openness, v, line_styles[k], linewidth=2, label=k)
+ plt.xlim(0, max(openness))
+ plt.ylim(ylim)
+ plt.xlabel('Openness (%)', fontsize=fontsize)
+ plt.ylabel('Open maF1 (%)', fontsize=fontsize)
+ plt.xticks(fontsize=fontsize)
+ plt.yticks(np.arange(ylim[0], ylim[1]+1, 5), fontsize=fontsize)
+ plt.grid('on')
+ plt.legend(fontsize=fontsize-10, loc='lower center', ncol=3, handletextpad=0.3, columnspacing=0.5)
+ plt.tight_layout()
+ result_path = os.path.dirname(result_prefix)
+ if not os.path.exists(result_path):
+ os.makedirs(result_path)
+ plt.savefig(result_prefix + '_%s.png'%(args.ood_data), bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ plt.savefig(result_prefix + '_%s.pdf'%(args.ood_data), bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+
+def main_i3d():
+ # SoftMax
+ print('Compute Open maF1 for SoftMax...')
+ result_file = 'i3d/results_baselines/openmax/I3D_OpenMax_%s_result.npz'%(args.ood_data)
+ openness_softmax, maF1_softmax = softmax_curvepoints(result_file, 0.996825, args.ood_ncls, args.num_rand)
+
+ # OpenMax
+ print('Compute Open maF1 for OpenMax...')
+ result_file = 'i3d/results_baselines/openmax/I3D_OpenMax_%s_result.npz'%(args.ood_data)
+ openness_openmax, maF1_openmax = openmax_curvepoints(result_file, args.ood_ncls, args.num_rand)
+
+ # RPL
+ print('Compute Open maF1 for RPL...')
+ result_file = 'i3d/results_baselines/rpl/I3D_RPL_%s_result.npz'%(args.ood_data)
+ openness_rpl, maF1_rpl = softmax_curvepoints(result_file, 0.995178, args.ood_ncls, args.num_rand)
+
+ # MCDropout BALD
+ print('Compute Open maF1 for MC Dropout BALD...')
+ result_file = 'i3d/results/I3D_DNN_BALD_%s_result.npz'%(args.ood_data)
+ openness_dnn, maF1_dnn = uncertainty_curvepoints(result_file, 0.000433, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # BNN SVI BALD
+ print('Compute Open maF1 for BNN SVI BALD...')
+ result_file = 'i3d/results/I3D_BNN_BALD_%s_result.npz'%(args.ood_data)
+ openness_bnn, maF1_bnn = uncertainty_curvepoints(result_file, 0.000004, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # DEAR (full)
+ print('Compute Open maF1 for DEAR (full)...')
+ result_file = 'i3d/results/I3D_EDLNoKLAvUCDebias_EDL_%s_result.npz'%(args.ood_data)
+ openness_dear, maF1_dear = uncertainty_curvepoints(result_file, 0.004550, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # draw F1 curve
+ line_styles = {'DEAR (full)': 'r-', 'SoftMax': 'b-', 'RPL': 'm-', 'BNN SVI': 'c-', 'MC Dropout': 'y-', 'OpenMax': 'k-'}
+ values = {'DEAR (full)': maF1_dear, 'SoftMax': maF1_softmax, 'RPL': maF1_rpl, 'BNN SVI': maF1_bnn, 'MC Dropout': maF1_dnn, 'OpenMax': maF1_openmax}
+ result_prefix = args.result_prefix + '_I3D'
+ plot_all_curves(openness_dear, values, line_styles, result_prefix, ylim=[60,80], fontsize=30)
+
+
+def main_tsm():
+ # SoftMax
+ print('Compute Open maF1 for SoftMax...')
+ result_file = 'tsm/results_baselines/openmax/TSM_OpenMax_%s_result.npz'%(args.ood_data)
+ openness_softmax, maF1_softmax = softmax_curvepoints(result_file, 0.999683, args.ood_ncls, args.num_rand)
+
+ # OpenMax
+ print('Compute Open maF1 for OpenMax...')
+ result_file = 'tsm/results_baselines/openmax/TSM_OpenMax_%s_result.npz'%(args.ood_data)
+ openness_openmax, maF1_openmax = openmax_curvepoints(result_file, args.ood_ncls, args.num_rand)
+
+ # RPL
+ print('Compute Open maF1 for RPL...')
+ result_file = 'tsm/results_baselines/rpl/TSM_RPL_%s_result.npz'%(args.ood_data)
+ openness_rpl, maF1_rpl = softmax_curvepoints(result_file, 0.999167, args.ood_ncls, args.num_rand)
+
+ # MCDropout BALD
+ print('Compute Open maF1 for MC Dropout BALD...')
+ result_file = 'tsm/results/TSM_DNN_BALD_%s_result.npz'%(args.ood_data)
+ openness_dnn, maF1_dnn = uncertainty_curvepoints(result_file, 0.000022, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # BNN SVI BALD
+ print('Compute Open maF1 for BNN SVI BALD...')
+ result_file = 'tsm/results/TSM_BNN_BALD_%s_result.npz'%(args.ood_data)
+ openness_bnn, maF1_bnn = uncertainty_curvepoints(result_file, 0.000003, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # DEAR (full)
+ print('Compute Open maF1 for DEAR (full)...')
+ result_file = 'tsm/results/TSM_EDLNoKLAvUCDebias_EDL_%s_result.npz'%(args.ood_data)
+ openness_dear, maF1_dear = uncertainty_curvepoints(result_file, 0.004549, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # draw F1 curve
+ line_styles = {'DEAR (full)': 'r-', 'SoftMax': 'b-', 'RPL': 'm-', 'BNN SVI': 'c-', 'MC Dropout': 'y-', 'OpenMax': 'k-'}
+ values = {'DEAR (full)': maF1_dear, 'SoftMax': maF1_softmax, 'RPL': maF1_rpl, 'BNN SVI': maF1_bnn, 'MC Dropout': maF1_dnn, 'OpenMax': maF1_openmax}
+ result_prefix = args.result_prefix + '_TSM'
+ ylim = [60, 90] if args.ood_data == 'HMDB' else [55, 90]
+ plot_all_curves(openness_dear, values, line_styles, result_prefix, ylim=ylim, fontsize=30)
+
+
+def main_slowfast():
+ # SoftMax
+ print('Compute Open maF1 for SoftMax...')
+ result_file = 'slowfast/results_baselines/openmax/SlowFast_OpenMax_%s_result.npz'%(args.ood_data)
+ openness_softmax, maF1_softmax = softmax_curvepoints(result_file, 0.997915, args.ood_ncls, args.num_rand)
+
+ # OpenMax
+ print('Compute Open maF1 for OpenMax...')
+ result_file = 'slowfast/results_baselines/openmax/SlowFast_OpenMax_%s_result.npz'%(args.ood_data)
+ openness_openmax, maF1_openmax = openmax_curvepoints(result_file, args.ood_ncls, args.num_rand)
+
+ # RPL
+ print('Compute Open maF1 for RPL...')
+ result_file = 'slowfast/results_baselines/rpl/SlowFast_RPL_%s_result.npz'%(args.ood_data)
+ openness_rpl, maF1_rpl = softmax_curvepoints(result_file, 0.997780, args.ood_ncls, args.num_rand)
+
+ # MCDropout BALD
+ print('Compute Open maF1 for MC Dropout BALD...')
+ result_file = 'slowfast/results/SlowFast_DNN_BALD_%s_result.npz'%(args.ood_data)
+ openness_dnn, maF1_dnn = uncertainty_curvepoints(result_file, 0.000065, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # BNN SVI BALD
+ print('Compute Open maF1 for BNN SVI BALD...')
+ result_file = 'slowfast/results/SlowFast_BNN_BALD_%s_result.npz'%(args.ood_data)
+ openness_bnn, maF1_bnn = uncertainty_curvepoints(result_file, 0.000004, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # DEAR (full)
+ print('Compute Open maF1 for DEAR (full)...')
+ result_file = 'slowfast/results/SlowFast_EDLNoKLAvUCDebias_EDL_%s_result.npz'%(args.ood_data)
+ openness_dear, maF1_dear = uncertainty_curvepoints(result_file, 0.004552, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # draw F1 curve
+ line_styles = {'DEAR (full)': 'r-', 'SoftMax': 'b-', 'RPL': 'm-', 'BNN SVI': 'c-', 'MC Dropout': 'y-', 'OpenMax': 'k-'}
+ values = {'DEAR (full)': maF1_dear, 'SoftMax': maF1_softmax, 'RPL': maF1_rpl, 'BNN SVI': maF1_bnn, 'MC Dropout': maF1_dnn, 'OpenMax': maF1_openmax}
+ result_prefix = args.result_prefix + '_SlowFast'
+ plot_all_curves(openness_dear, values, line_styles, result_prefix, ylim=[60,90], fontsize=30)
+
+
+
+def main_tpn():
+ # SoftMax
+ print('Compute Open maF1 for SoftMax...')
+ result_file = 'tpn_slowonly/results_baselines/openmax/TPN_OpenMax_%s_result.npz'%(args.ood_data)
+ openness_softmax, maF1_softmax = softmax_curvepoints(result_file, 0.997623, args.ood_ncls, args.num_rand)
+
+ # OpenMax
+ print('Compute Open maF1 for OpenMax...')
+ result_file = 'tpn_slowonly/results_baselines/openmax/TPN_OpenMax_%s_result.npz'%(args.ood_data)
+ openness_openmax, maF1_openmax = openmax_curvepoints(result_file, args.ood_ncls, args.num_rand)
+
+ # RPL
+ print('Compute Open maF1 for RPL...')
+ result_file = 'tpn_slowonly/results_baselines/rpl/TPN_RPL_%s_result.npz'%(args.ood_data)
+ openness_rpl, maF1_rpl = softmax_curvepoints(result_file, 0.996931, args.ood_ncls, args.num_rand)
+
+ # MCDropout BALD
+ print('Compute Open maF1 for MC Dropout BALD...')
+ result_file = 'tpn_slowonly/results/TPN_SlowOnly_Dropout_BALD_%s_result.npz'%(args.ood_data)
+ openness_dnn, maF1_dnn = uncertainty_curvepoints(result_file, 0.000096, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # BNN SVI BALD
+ print('Compute Open maF1 for BNN SVI BALD...')
+ result_file = 'tpn_slowonly/results/TPN_SlowOnly_BNN_BALD_%s_result.npz'%(args.ood_data)
+ openness_bnn, maF1_bnn = uncertainty_curvepoints(result_file, 0.000007, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # DEAR (full)
+ print('Compute Open maF1 for DEAR (full)...')
+ result_file = 'tpn_slowonly/results/TPN_SlowOnly_EDLlogNoKLAvUCDebias_EDL_%s_result.npz'%(args.ood_data)
+ openness_dear, maF1_dear = uncertainty_curvepoints(result_file, 0.004555, args.ind_ncls, args.ood_ncls, args.num_rand)
+
+ # draw F1 curve
+ line_styles = {'DEAR (full)': 'r-', 'SoftMax': 'b-', 'RPL': 'm-', 'BNN SVI': 'c-', 'MC Dropout': 'y-', 'OpenMax': 'k-'}
+ values = {'DEAR (full)': maF1_dear, 'SoftMax': maF1_softmax, 'RPL': maF1_rpl, 'BNN SVI': maF1_bnn, 'MC Dropout': maF1_dnn, 'OpenMax': maF1_openmax}
+ result_prefix = args.result_prefix + '_TPN'
+ ylim = [50, 85] if args.ood_data == 'HMDB' else [50, 85]
+ plot_all_curves(openness_dear, values, line_styles, result_prefix, ylim=ylim, fontsize=30)
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='Compare the performance of Open macroF1 against openness')
+ # model config
+ parser.add_argument('--ind_ncls', type=int, default=101, help='the number of classes in known dataset')
+ parser.add_argument('--ood_ncls', type=int, default=51, choices=[51, 305], help='the number of classes in unknwon dataset')
+ parser.add_argument('--ood_data', default='HMDB', choices=['HMDB', 'MiT'], help='the name of OOD dataset.')
+ parser.add_argument('--model', default='I3D', choices=['I3D', 'TSM', 'SlowFast', 'TPN'], help='the action recognition model.')
+ parser.add_argument('--num_rand', type=int, default=10, help='the number of random selection for ood classes')
+ parser.add_argument('--result_prefix', default='../temp/F1_openness')
+ args = parser.parse_args()
+ return args
+
+if __name__ == '__main__':
+ """ Example script:
+ python draw_openness_curves.py --model I3D --ood_data MiT --ood_ncls 305
+ """
+ np.random.seed(123)
+ args = parse_args()
+
+ if args.model == 'I3D':
+ # draw results on I3D
+ main_i3d()
+ elif args.model == 'TSM':
+ # draw results on TSM
+ main_tsm()
+ elif args.model == 'SlowFast':
+ # draw results on SlowFast
+ main_slowfast()
+ elif args.model == 'TPN':
+ # draw results on TPN
+ main_tpn()
+ else:
+ raise NotImplementedError
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/draw_performance_gain.py b/Downstream/Open-Set-Action-Recognition/experiments/draw_performance_gain.py
new file mode 100644
index 0000000..19cf6a1
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/draw_performance_gain.py
@@ -0,0 +1,298 @@
+import matplotlib.pyplot as plt
+import numpy as np
+from matplotlib.lines import Line2D
+
+
+def draw_curves():
+ fig = plt.figure(figsize=(8,5))
+ plt.rcParams["font.family"] = "Arial"
+ fontsize = 15
+ markersize = 80
+
+ # I3D
+ I3D_DNN_HMDB = [94.69, 75.07] # (closed-set ACC, open-set AUC)
+ I3D_DNN_MiT = [94.69, 79.14]
+ I3D_DEAR_HMDB = [94.34, 77.08]
+ I3D_DEAR_MiT = [94.34, 81.54]
+ # TSM
+ TSM_DNN_HMDB = [95.11, 73.85]
+ TSM_DNN_MiT = [95.11, 78.35]
+ TSM_DEAR_HMDB = [94.45, 78.65]
+ TSM_DEAR_MiT = [94.45, 83.92]
+ # TPN
+ TPN_DNN_HMDB = [95.41, 74.13]
+ TPN_DNN_MiT = [95.41, 77.76]
+ TPN_DEAR_HMDB = [96.42, 79.23]
+ TPN_DEAR_MiT = [96.42, 81.80]
+ # SlowFast
+ SlowFast_DNN_HMDB = [96.72, 75.41]
+ SlowFast_DNN_MiT = [96.72, 78.49]
+ SlowFast_DEAR_HMDB = [96.66, 82.94]
+ SlowFast_DEAR_MiT = [96.66, 86.99]
+
+ # Line: DNN for HMDB
+ plt.plot([I3D_DNN_HMDB[0], TSM_DNN_HMDB[0], TPN_DNN_HMDB[0], SlowFast_DNN_HMDB[0]],
+ [I3D_DNN_HMDB[1], TSM_DNN_HMDB[1], TPN_DNN_HMDB[1], SlowFast_DNN_HMDB[1]], 'r-', linewidth=2, label='HMDB')
+ # Line: DEAR for HMDB
+ plt.plot([I3D_DEAR_HMDB[0], TSM_DEAR_HMDB[0], TPN_DEAR_HMDB[0], SlowFast_DEAR_HMDB[0]],
+ [I3D_DEAR_HMDB[1], TSM_DEAR_HMDB[1], TPN_DEAR_HMDB[1], SlowFast_DEAR_HMDB[1]], 'r-', linewidth=2)
+ # Line: DNN for MiT
+ plt.plot([I3D_DNN_MiT[0], TSM_DNN_MiT[0], TPN_DNN_MiT[0], SlowFast_DNN_MiT[0]],
+ [I3D_DNN_MiT[1], TSM_DNN_MiT[1], TPN_DNN_MiT[1], SlowFast_DNN_MiT[1]], 'b-', linewidth=2, label='MiT')
+ # Line: DEAR for MiT
+ plt.plot([I3D_DEAR_MiT[0], TSM_DEAR_MiT[0], TPN_DEAR_MiT[0], SlowFast_DEAR_MiT[0]],
+ [I3D_DEAR_MiT[1], TSM_DEAR_MiT[1], TPN_DEAR_MiT[1], SlowFast_DEAR_MiT[1]], 'b-', linewidth=2)
+
+
+ # Draw all I3D points
+ # HMDB
+ plt.scatter(I3D_DNN_HMDB[0], I3D_DNN_HMDB[1], marker='^', s=markersize, color='r', label='Dropout BALD')
+ plt.text(I3D_DNN_HMDB[0], I3D_DNN_HMDB[1], 'I3D', fontsize=fontsize)
+ plt.scatter(I3D_DEAR_HMDB[0], I3D_DEAR_HMDB[1], marker='*', s=markersize, color='r', label='DEAR EU')
+ plt.text(I3D_DEAR_HMDB[0], I3D_DEAR_HMDB[1], 'I3D', fontsize=fontsize)
+ plt.plot([I3D_DNN_HMDB[0], I3D_DEAR_HMDB[0]], [I3D_DNN_HMDB[1], I3D_DEAR_HMDB[1]], 'r--', linewidth=0.5)
+ # plt.arrow(I3D_DNN_HMDB[0]+1, I3D_DNN_HMDB[1], I3D_DEAR_HMDB[0]-I3D_DNN_HMDB[0]-2, I3D_DEAR_HMDB[1]-I3D_DNN_HMDB[1]-1,head_width=0.8, fc='skyblue',ec='skyblue', head_length=0.8)
+ # # MiT
+ plt.scatter(I3D_DNN_MiT[0], I3D_DNN_MiT[1], marker='^', s=markersize, color='b')
+ plt.text(I3D_DNN_MiT[0], I3D_DNN_MiT[1], 'I3D', fontsize=fontsize)
+ plt.scatter(I3D_DEAR_MiT[0], I3D_DEAR_MiT[1], marker='*', s=markersize, color='b')
+ plt.text(I3D_DEAR_MiT[0], I3D_DEAR_MiT[1], 'I3D', fontsize=fontsize)
+ plt.plot([I3D_DNN_MiT[0], I3D_DEAR_MiT[0]], [I3D_DNN_MiT[1], I3D_DEAR_MiT[1]], 'b--', linewidth=0.5)
+ # plt.arrow(I3D_DNN_MiT[0]+1, I3D_DNN_MiT[1], I3D_DEAR_MiT[0]-I3D_DNN_MiT[0]-3, I3D_DEAR_MiT[1]-I3D_DNN_MiT[1]-2,head_width=0.8, fc='grey',ec='grey', head_length=0.8)
+
+ # Draw all TSM points
+ # HMDB
+ plt.scatter(TSM_DNN_HMDB[0], TSM_DNN_HMDB[1], marker='^', s=markersize, color='r')
+ plt.text(TSM_DNN_HMDB[0], TSM_DNN_HMDB[1], 'TSM', fontsize=fontsize)
+ plt.scatter(TSM_DEAR_HMDB[0], TSM_DEAR_HMDB[1], marker='*', s=markersize, color='r')
+ plt.text(TSM_DEAR_HMDB[0], TSM_DEAR_HMDB[1], 'TSM', fontsize=fontsize)
+ plt.plot([TSM_DNN_HMDB[0], TSM_DEAR_HMDB[0]], [TSM_DNN_HMDB[1], TSM_DEAR_HMDB[1]], 'r--', linewidth=0.5)
+ # plt.arrow(TSM_DNN_HMDB[0]+1, TSM_DNN_HMDB[1], TSM_DEAR_HMDB[0]-TSM_DNN_HMDB[0]-2, TSM_DEAR_HMDB[1]-TSM_DNN_HMDB[1]-1,head_width=0.8, fc='skyblue',ec='skyblue', head_length=0.8)
+ # # MiT
+ plt.scatter(TSM_DNN_MiT[0], TSM_DNN_MiT[1], marker='^', s=markersize, color='b')
+ plt.text(TSM_DNN_MiT[0], TSM_DNN_MiT[1], 'TSM', fontsize=fontsize)
+ plt.scatter(TSM_DEAR_MiT[0], TSM_DEAR_MiT[1], marker='*', s=markersize, color='b')
+ plt.text(TSM_DEAR_MiT[0], TSM_DEAR_MiT[1], 'TSM', fontsize=fontsize)
+ plt.plot([TSM_DNN_MiT[0], TSM_DEAR_MiT[0]], [TSM_DNN_MiT[1], TSM_DEAR_MiT[1]], 'b--', linewidth=0.5)
+
+ # Draw all TPN points
+ # HMDB
+ plt.scatter(TPN_DNN_HMDB[0], TPN_DNN_HMDB[1], marker='^', s=markersize, color='r')
+ plt.text(TPN_DNN_HMDB[0], TPN_DNN_HMDB[1], 'TPN', fontsize=fontsize)
+ plt.scatter(TPN_DEAR_HMDB[0], TPN_DEAR_HMDB[1], marker='*', s=markersize, color='r')
+ plt.text(TPN_DEAR_HMDB[0], TPN_DEAR_HMDB[1], 'TPN', fontsize=fontsize)
+ plt.plot([TPN_DNN_HMDB[0], TPN_DEAR_HMDB[0]], [TPN_DNN_HMDB[1], TPN_DEAR_HMDB[1]], 'r--', linewidth=0.5)
+ # plt.arrow(TPN_DNN_HMDB[0]+1, TPN_DNN_HMDB[1], TPN_DEAR_HMDB[0]-TPN_DNN_HMDB[0]-2, TPN_DEAR_HMDB[1]-TPN_DNN_HMDB[1]-1,head_width=0.8, fc='skyblue',ec='skyblue', head_length=0.8)
+ plt.scatter(TPN_DNN_MiT[0], TPN_DNN_MiT[1], marker='^', s=markersize, color='b')
+ plt.text(TPN_DNN_MiT[0], TPN_DNN_MiT[1], 'TPN', fontsize=fontsize)
+ plt.scatter(TPN_DEAR_MiT[0], TPN_DEAR_MiT[1], marker='*', s=markersize, color='b')
+ plt.text(TPN_DEAR_MiT[0], TPN_DEAR_MiT[1], 'TPN', fontsize=fontsize)
+ plt.plot([TPN_DNN_MiT[0], TPN_DEAR_MiT[0]], [TPN_DNN_MiT[1], TPN_DEAR_MiT[1]], 'b--', linewidth=0.5)
+
+ # Draw all SlowFast points
+ # HMDB
+ plt.scatter(SlowFast_DNN_HMDB[0], SlowFast_DNN_HMDB[1], marker='^', s=markersize, color='r')
+ plt.text(SlowFast_DNN_HMDB[0], SlowFast_DNN_HMDB[1], 'SlowFast', fontsize=fontsize)
+ plt.scatter(SlowFast_DEAR_HMDB[0], SlowFast_DEAR_HMDB[1], marker='*', s=markersize, color='r')
+ plt.text(SlowFast_DEAR_HMDB[0], SlowFast_DEAR_HMDB[1], 'SlowFast', fontsize=fontsize)
+ plt.plot([SlowFast_DNN_HMDB[0], SlowFast_DEAR_HMDB[0]], [SlowFast_DNN_HMDB[1], SlowFast_DEAR_HMDB[1]], 'r--', linewidth=0.5)
+ # MiT
+ plt.scatter(SlowFast_DNN_MiT[0], SlowFast_DNN_MiT[1], marker='^', s=markersize, color='b')
+ plt.text(SlowFast_DNN_MiT[0], SlowFast_DNN_MiT[1], 'SlowFast', fontsize=fontsize)
+ plt.scatter(SlowFast_DEAR_MiT[0], SlowFast_DEAR_MiT[1], marker='*', s=markersize, color='b')
+ plt.text(SlowFast_DEAR_MiT[0], SlowFast_DEAR_MiT[1], 'SlowFast', fontsize=fontsize)
+ plt.plot([SlowFast_DNN_MiT[0], SlowFast_DEAR_MiT[0]], [SlowFast_DNN_MiT[1], SlowFast_DEAR_MiT[1]], 'b--', linewidth=0.5)
+
+ plt.xlim(94, 97.5)
+ plt.ylim(65, 90)
+ plt.xlabel('Closed-Set Accuracy (%)', fontsize=fontsize)
+ plt.ylabel('Open-Set AUC Score (%)', fontsize=fontsize)
+ plt.xticks(fontsize=fontsize)
+ plt.yticks(fontsize=fontsize)
+ plt.legend(loc='lower left', fontsize=fontsize)
+ plt.grid('on', linestyle='--')
+ plt.tight_layout()
+
+ plt.savefig('../temp/compare_gain.png')
+ plt.savefig('../temp/compare_gain.pdf')
+
+
+def draw_one_curve(data_dict, markers, markercolor='g', markersize=80, fontsize=10, label='I3D', linestyle='g-', add_marker_text=False, text_offset=[0,0]):
+
+ sorted_dict = dict(sorted(data_dict.items(),key=lambda x:x[1][1]))
+
+ x_data, y_data = [], []
+ for k, v in sorted_dict.items():
+ x_data.append(v[1])
+ y_data.append(v[0])
+ # Marker: OpenMax
+ if k == 'DEAR (Ours)':
+ plt.scatter(v[1], v[0], marker=markers[k], s=markersize*4, color=markercolor)
+ else:
+ plt.scatter(v[1], v[0], marker=markers[k], s=markersize, color=markercolor)
+ if add_marker_text:
+ # plt.text(v[1] + text_offset[0], v[0]+ text_offset[1], k, fontsize=fontsize)
+ pass
+ # Line: I3D for MiT
+ line_hd, = plt.plot(x_data, y_data, linestyle, linewidth=2, label=label, markersize=1)
+ return line_hd
+
+
+def draw_mit_curves():
+ fig, ax = plt.subplots(figsize=(8,6))
+ plt.rcParams["font.family"] = "Arial"
+ fontsize = 25
+ markersize = 80
+
+ # (open maF1, open-set AUC)
+ # I3D
+ I3D_OpenMax = [66.22, 77.76]
+ I3D_Dropout = [68.11, 79.14]
+ I3D_BNNSVI = [68.65, 79.50]
+ I3D_SoftMax = [68.84, 79.94]
+ I3D_RPL = [68.11, 79.16]
+ I3D_DEAR = [69.98, 81.54]
+ # TSM
+ TSM_OpenMax = [71.81, 83.05]
+ TSM_Dropout = [65.32, 78.35]
+ TSM_BNNSVI = [64.28, 77.39]
+ TSM_SoftMax = [71.68, 82.38]
+ TSM_RPL = [63.92, 77.28]
+ TSM_DEAR = [70.15, 83.92]
+ # TPN
+ TPN_OpenMax = [64.80, 76.26]
+ TPN_Dropout = [65.77, 77.76]
+ TPN_BNNSVI = [61.40, 75.32]
+ TPN_SoftMax = [70.82, 81.35]
+ TPN_RPL = [66.21, 78.21]
+ TPN_DEAR = [71.18, 81.80]
+ # SlowFast
+ SlowFast_OpenMax = [72.48, 80.62]
+ SlowFast_Dropout = [67.53, 78.49]
+ SlowFast_BNNSVI = [65.22, 77.39]
+ SlowFast_SoftMax = [74.42, 82.88]
+ SlowFast_RPL = [66.33, 77.42]
+ SlowFast_DEAR = [77.28, 86.99]
+
+ markers = {'DEAR (Ours)': '*', 'SoftMax': 'o', 'OpenMax': '^', 'RPL': 'd', 'MC Dropout': 's', 'BNN SVI': 'P'}
+
+ # Line: I3D for MiT
+ data_dict = {'OpenMax': I3D_OpenMax, 'MC Dropout': I3D_Dropout, 'BNN SVI': I3D_BNNSVI, 'SoftMax': I3D_SoftMax, 'RPL': I3D_RPL, 'DEAR (Ours)': I3D_DEAR}
+ line1_hd = draw_one_curve(data_dict, markers=markers, markercolor='g', markersize=markersize, fontsize=fontsize, label='I3D', linestyle='g-')
+
+ data_dict = {'OpenMax': TSM_OpenMax, 'MC Dropout': TSM_Dropout, 'BNN SVI': TSM_BNNSVI, 'SoftMax': TSM_SoftMax, 'RPL': TSM_RPL, 'DEAR (Ours)': TSM_DEAR}
+ line2_hd = draw_one_curve(data_dict, markers=markers, markercolor='k', markersize=markersize, fontsize=fontsize, label='TSM', linestyle='k-')
+
+ data_dict = {'OpenMax': TPN_OpenMax, 'MC Dropout': TPN_Dropout, 'BNN SVI': TPN_BNNSVI, 'SoftMax': TPN_SoftMax, 'RPL': TPN_RPL, 'DEAR (Ours)': TPN_DEAR}
+ line3_hd = draw_one_curve(data_dict, markers=markers, markercolor='b', markersize=markersize, fontsize=fontsize, label='TPN', linestyle='b-')
+
+ data_dict = {'OpenMax': SlowFast_OpenMax, 'MC Dropout': SlowFast_Dropout, 'BNN SVI': SlowFast_BNNSVI, 'SoftMax': SlowFast_SoftMax, 'RPL': SlowFast_RPL, 'DEAR (Ours)': SlowFast_DEAR}
+ line4_hd = draw_one_curve(data_dict, markers=markers, markercolor='r', markersize=markersize, fontsize=fontsize, label='SlowFast', linestyle='r-',
+ add_marker_text=True, text_offset=[-2.2, -0.2])
+
+ marker_elements = []
+ for k, v in markers.items():
+ msize = 18 if k == 'DEAR (Ours)' else 12
+ elem = Line2D([0], [0], marker=v, label=k, markersize=msize, linestyle="None")
+ marker_elements.append(elem)
+ marker_legend = ax.legend(handles=marker_elements, fontsize=fontsize-3, loc='lower right', ncol=1, handletextpad=0.05, columnspacing=0.05, borderaxespad=0.1)
+ ax.add_artist(marker_legend)
+
+ plt.ylim(60, 78)
+ plt.xlim(75, 88)
+ plt.ylabel('Open maF1 (%)', fontsize=fontsize)
+ plt.xlabel('Open-Set AUC Score (%)', fontsize=fontsize)
+ plt.xticks(np.arange(75, 89, 4), fontsize=fontsize)
+ plt.yticks(np.arange(60, 79, 4), fontsize=fontsize)
+ plt.legend(handles=[line1_hd, line2_hd, line3_hd, line4_hd], loc='upper left', fontsize=fontsize-3, handletextpad=0.5, borderaxespad=0.1)
+ plt.title('MiT-v2 as Unknown', fontsize=fontsize)
+ plt.grid('on', linestyle='--')
+ plt.tight_layout()
+ plt.savefig('../temp/compare_gain_mit.png', bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ plt.savefig('../temp/compare_gain_mit.pdf', bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+
+
+
+def draw_hmdb_curves():
+ fig, ax = plt.subplots(figsize=(8,6))
+ plt.rcParams["font.family"] = "Arial"
+ fontsize = 25
+ markersize = 80
+
+ # (open maF1, open-set AUC)
+ # I3D
+ I3D_OpenMax = [67.85, 74.34]
+ I3D_Dropout = [71.13, 75.07]
+ I3D_BNNSVI = [71.57, 74.66]
+ I3D_SoftMax = [73.19, 75.68]
+ I3D_RPL = [71.48, 75.20]
+ I3D_DEAR = [77.24, 77.08]
+ # TSM
+ TSM_OpenMax = [74.17, 77.07]
+ TSM_Dropout = [71.52, 73.85]
+ TSM_BNNSVI = [69.11, 73.42]
+ TSM_SoftMax = [78.27, 77.99]
+ TSM_RPL = [69.34, 73.62]
+ TSM_DEAR = [84.69, 78.65]
+ # TPN
+ TPN_OpenMax = [65.27, 74.12]
+ TPN_Dropout = [68.45, 74.13]
+ TPN_BNNSVI = [63.81, 72.68]
+ TPN_SoftMax = [76.23, 77.97]
+ TPN_RPL = [70.31, 75.32]
+ TPN_DEAR = [81.79, 79.23]
+ # SlowFast
+ SlowFast_OpenMax = [73.57, 78.76]
+ SlowFast_Dropout = [70.55, 75.41]
+ SlowFast_BNNSVI = [69.19, 74.78]
+ SlowFast_SoftMax = [78.04, 79.16]
+ SlowFast_RPL = [68.32, 74.23]
+ SlowFast_DEAR = [85.48, 82.94]
+
+ markers = {'DEAR (Ours)': '*', 'SoftMax': 'o', 'OpenMax': '^', 'RPL': 'd', 'MC Dropout': 's', 'BNN SVI': 'P'}
+
+ # Line: I3D for HMDB
+ data_dict = {'OpenMax': I3D_OpenMax, 'MC Dropout': I3D_Dropout, 'BNN SVI': I3D_BNNSVI, 'SoftMax': I3D_SoftMax, 'RPL': I3D_RPL, 'DEAR (Ours)': I3D_DEAR}
+ line1_hd = draw_one_curve(data_dict, markers=markers, markercolor='g', markersize=markersize, fontsize=fontsize, label='I3D', linestyle='g-')
+
+ data_dict = {'OpenMax': TSM_OpenMax, 'MC Dropout': TSM_Dropout, 'BNN SVI': TSM_BNNSVI, 'SoftMax': TSM_SoftMax, 'RPL': TSM_RPL, 'DEAR (Ours)': TSM_DEAR}
+ line2_hd = draw_one_curve(data_dict, markers=markers, markercolor='k', markersize=markersize, fontsize=fontsize, label='TSM', linestyle='k-')
+
+ data_dict = {'OpenMax': TPN_OpenMax, 'MC Dropout': TPN_Dropout, 'BNN SVI': TPN_BNNSVI, 'SoftMax': TPN_SoftMax, 'RPL': TPN_RPL, 'DEAR (Ours)': TPN_DEAR}
+ line3_hd = draw_one_curve(data_dict, markers=markers, markercolor='b', markersize=markersize, fontsize=fontsize, label='TPN', linestyle='b-')
+
+ data_dict = {'OpenMax': SlowFast_OpenMax, 'MC Dropout': SlowFast_Dropout, 'BNN SVI': SlowFast_BNNSVI, 'SoftMax': SlowFast_SoftMax, 'RPL': SlowFast_RPL, 'DEAR (Ours)': SlowFast_DEAR}
+ line4_hd = draw_one_curve(data_dict, markers=markers, markercolor='r', markersize=markersize, fontsize=fontsize, label='SlowFast', linestyle='r-',
+ add_marker_text=True, text_offset=[0.2, -1.5])
+
+
+ marker_elements = []
+ for k, v in markers.items():
+ msize = 18 if k == 'DEAR (Ours)' else 12
+ elem = Line2D([0], [0], marker=v, label=k, markersize=msize, linestyle="None")
+ marker_elements.append(elem)
+ marker_legend = ax.legend(handles=marker_elements, fontsize=fontsize-3, loc='lower right', ncol=1, handletextpad=0.3, columnspacing=0.2, borderaxespad=0.1)
+ ax.add_artist(marker_legend)
+
+ plt.ylim(60, 88)
+ plt.xlim(72, 85)
+ plt.ylabel('Open maF1 (%)', fontsize=fontsize)
+ plt.xlabel('Open-Set AUC Score (%)', fontsize=fontsize)
+ plt.xticks(fontsize=fontsize)
+ plt.yticks(fontsize=fontsize)
+ plt.legend(handles=[line1_hd, line2_hd, line3_hd, line4_hd], loc='upper left', fontsize=fontsize-3, handletextpad=0.5, borderaxespad=0.1)
+ plt.grid('on', linestyle='--')
+ plt.title('HMDB-51 as Unknown', fontsize=fontsize)
+ plt.tight_layout()
+ plt.savefig('../temp/compare_gain_hmdb.png', bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+ plt.savefig('../temp/compare_gain_hmdb.pdf', bbox_inches='tight', dpi=fig.dpi, pad_inches=0.0)
+
+
+if __name__ == '__main__':
+
+ models = ['I3D', 'TPN', 'TSM', 'SlowFast']
+
+ # draw_curves()
+
+ draw_mit_curves()
+ draw_hmdb_curves()
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/eval_debias.py b/Downstream/Open-Set-Action-Recognition/experiments/eval_debias.py
new file mode 100644
index 0000000..1aaeffb
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/eval_debias.py
@@ -0,0 +1,129 @@
+import argparse
+import os
+import torch
+import mmcv
+from mmaction.apis import init_recognizer
+from mmaction.datasets import build_dataloader, build_dataset
+from mmaction.core.evaluation import top_k_accuracy
+from mmcv.parallel import MMDataParallel
+import numpy as np
+import torch.multiprocessing
+torch.multiprocessing.set_sharing_strategy('file_system')
+
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='MMAction2 test')
+ # model config
+ parser.add_argument('config', help='test config file path')
+ parser.add_argument('checkpoint', help='checkpoint file/url')
+ parser.add_argument('--split_file', help='the split file for evaluation')
+ parser.add_argument('--video_path', help='the video path for evaluation')
+ parser.add_argument('--device', type=str, default='cuda:0', help='CPU/CUDA device option')
+ parser.add_argument('--result_prefix', help='result file prefix')
+ args = parser.parse_args()
+ return args
+
+
+def set_deterministic(seed):
+ torch.manual_seed(seed)
+ torch.cuda.manual_seed(seed)
+ torch.cuda.manual_seed_all(seed) # if you are using multi-GPU.
+ np.random.seed(seed) # Numpy module.
+ torch.backends.cudnn.benchmark = True
+ torch.backends.cudnn.deterministic = True
+
+
+def init_inference():
+ # build the recognizer from a config file and checkpoint file/url
+ model = init_recognizer(args.config, args.checkpoint, device=device, use_frames=False)
+ cfg = model.cfg
+ cfg.data.test.test_mode = True
+ cfg.data.test.ann_file = args.split_file
+ cfg.data.test.data_prefix = args.video_path
+ evidence = cfg.get('evidence', 'exp')
+ # build the dataloader
+ dataset = build_dataset(cfg.data.test, dict(test_mode=True))
+ dataloader_setting = dict(
+ videos_per_gpu=cfg.data.get('videos_per_gpu', 1),
+ workers_per_gpu=cfg.data.get('workers_per_gpu', 1),
+ dist=False,
+ shuffle=False,
+ pin_memory=False)
+ dataloader_setting = dict(dataloader_setting, **cfg.data.get('test_dataloader', {}))
+ data_loader = build_dataloader(dataset, **dataloader_setting)
+ return model, data_loader, evidence
+
+
+def run_evidence_inference(model, data_loader, evidence='exp'):
+ # get the evidence function
+ if evidence == 'relu':
+ from mmaction.models.losses.edl_loss import relu_evidence as get_evidence
+ elif evidence == 'exp':
+ from mmaction.models.losses.edl_loss import exp_evidence as get_evidence
+ elif evidence == 'softplus':
+ from mmaction.models.losses.edl_loss import softplus_evidence as get_evidence
+ else:
+ raise NotImplementedError
+ num_classes = model.cls_head.num_classes
+
+ # run inference
+ model = MMDataParallel(model, device_ids=[0])
+ all_scores, all_uncertainties, all_labels = [], [], []
+ prog_bar = mmcv.ProgressBar(len(data_loader.dataset))
+ for i, data in enumerate(data_loader):
+ with torch.no_grad():
+ output = model(return_loss=False, **data)
+ evidence = get_evidence(torch.from_numpy(output))
+ alpha = evidence + 1
+ uncertainty = num_classes / torch.sum(alpha, dim=1)
+ scores = alpha / torch.sum(alpha, dim=1, keepdim=True)
+ all_uncertainties.append(uncertainty.numpy())
+ all_scores.append(scores.numpy())
+ labels = data['label'].numpy()
+ all_labels.append(labels)
+
+ # use the first key as main key to calculate the batch size
+ batch_size = len(next(iter(data.values())))
+ for _ in range(batch_size):
+ prog_bar.update()
+ all_scores = np.concatenate(all_scores, axis=0)
+ all_uncertainties = np.concatenate(all_uncertainties, axis=0)
+ all_labels = np.concatenate(all_labels, axis=0)
+
+ return all_scores, all_uncertainties, all_labels
+
+
+def main():
+
+ result_file = os.path.join(args.result_prefix + '_result.npz')
+ if not os.path.exists(result_file):
+ # prepare result path
+ result_dir = os.path.dirname(result_file)
+ if not os.path.exists(result_dir):
+ os.makedirs(result_dir)
+ # init
+ model, data_loader, evidence = init_inference()
+ # run inference
+ all_scores, all_uncertainties, all_labels = run_evidence_inference(model, data_loader, evidence)
+ # save
+ np.savez(result_file[:-4], score=all_scores, uncertainty=all_uncertainties, label=all_labels)
+ else:
+ results = np.load(result_file, allow_pickle=True)
+ all_scores = results['score']
+ all_uncertainties = results['uncertainty']
+ all_labels = results['label'] # (N2,)
+
+ # evaluation on bias/unbiased closed set
+ top_k_acc = top_k_accuracy(all_scores, all_labels, (1, 5))
+ print('Evaluation Results:\ntop1_acc: %.5lf, top5_acc: %.5lf\n'%(top_k_acc[0], top_k_acc[1]))
+
+
+if __name__ == '__main__':
+
+ args = parse_args()
+ # assign the desired device.
+ device = torch.device(args.device)
+ set_deterministic(1234)
+
+ main()
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/evaluate_calibration.py b/Downstream/Open-Set-Action-Recognition/experiments/evaluate_calibration.py
new file mode 100644
index 0000000..f9b8c8c
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/evaluate_calibration.py
@@ -0,0 +1,158 @@
+import argparse, os
+import numpy as np
+import matplotlib.pyplot as plt
+
+
+def eval_calibration(predictions, confidences, labels, M=15):
+ """
+ M: number of bins for confidence scores
+ """
+ num_Bm = np.zeros((M,), dtype=np.int32)
+ accs = np.zeros((M,), dtype=np.float32)
+ confs = np.zeros((M,), dtype=np.float32)
+ for m in range(M):
+ interval = [m / M, (m+1) / M]
+ Bm = np.where((confidences > interval[0]) & (confidences <= interval[1]))[0]
+ if len(Bm) > 0:
+ acc_bin = np.sum(predictions[Bm] == labels[Bm]) / len(Bm)
+ conf_bin = np.mean(confidences[Bm])
+ # gather results
+ num_Bm[m] = len(Bm)
+ accs[m] = acc_bin
+ confs[m] = conf_bin
+ conf_intervals = np.arange(0, 1, 1/M)
+ return accs, confs, num_Bm, conf_intervals
+
+def add_identity(axes, *line_args, **line_kwargs):
+ identity, = axes.plot([], [], *line_args, **line_kwargs)
+ def callback(axes):
+ low_x, high_x = axes.get_xlim()
+ low_y, high_y = axes.get_ylim()
+ low = max(low_x, low_y)
+ high = min(high_x, high_y)
+ identity.set_data([low, high], [low, high])
+ callback(axes)
+ axes.callbacks.connect('xlim_changed', callback)
+ axes.callbacks.connect('ylim_changed', callback)
+ return axes
+
+
+def compute_eavuc(preds, labels, confs, uncertainties):
+ eavuc = 0
+ inds_accurate = np.where(preds == labels)[0]
+ eavuc += -np.sum(confs[inds_accurate] * np.log(1 - uncertainties[inds_accurate]))
+ inds_inaccurate = np.where(preds != labels)[0]
+ eavuc += -np.sum((1 - confs[inds_inaccurate]) * np.log(uncertainties[inds_inaccurate]))
+ return eavuc
+
+
+def closedset_multicls(ind_results, ind_labels, ind_confidences, ind_uncertainties):
+ ind_preds = ind_results.copy()
+ accs, confs, num_Bm, conf_intervals = eval_calibration(ind_preds, 1-ind_uncertainties, ind_labels, M=args.M)
+ # compute the EAvUC
+ eavuc = compute_eavuc(ind_preds, ind_labels, ind_confidences, ind_uncertainties)
+ # compute ECE
+ ece = np.sum(np.abs(accs - confs) * num_Bm / np.sum(num_Bm))
+ return ece, eavuc, conf_intervals, accs
+
+
+def openset_multicls(ind_results, ood_results, ind_labels, ood_labels, ind_confidences, ood_confidences, ind_uncertainties, ood_uncertainties):
+ ind_preds = ind_results.copy()
+ ood_preds = ood_results.copy()
+ ind_preds[ind_uncertainties > args.threshold] = args.ind_ncls
+ ood_preds[ood_uncertainties > args.threshold] = args.ind_ncls
+ preds = np.concatenate((ind_preds, ood_preds), axis=0)
+ labels = np.concatenate((ind_labels, np.ones_like(ood_labels) * args.ind_ncls), axis=0)
+ confs = np.concatenate((ind_confidences, ood_confidences), axis=0)
+ unctns = np.concatenate((ind_uncertainties, ood_uncertainties), axis=0)
+ # compute the EAvUC
+ eavuc = compute_eavuc(preds, labels, confs, unctns)
+ # compute ECE
+ accs, confs, num_Bm, conf_intervals = eval_calibration(preds, 1-unctns, labels, M=args.M)
+ ece = np.sum(np.abs(accs - confs) * num_Bm / np.sum(num_Bm))
+ return ece, eavuc, conf_intervals, accs
+
+
+def openset_bincls(ind_results, ood_results, ind_labels, ood_labels, ind_confidences, ood_confidences, ind_uncertainties, ood_uncertainties):
+ ind_preds = ind_results.copy()
+ ood_preds = ood_results.copy()
+ ind_preds[ind_uncertainties > args.threshold] = 1
+ ind_preds[ind_uncertainties <= args.threshold] = 0
+ ood_preds[ood_uncertainties > args.threshold] = 1
+ ood_preds[ood_uncertainties < args.threshold] = 0
+ preds = np.concatenate((ind_preds, ood_preds), axis=0)
+ labels = np.concatenate((np.zeros_like(ind_labels), np.ones_like(ood_labels)), axis=0)
+ confs = np.concatenate((ind_confidences, ood_confidences), axis=0)
+ unctns = np.concatenate((ind_uncertainties, ood_uncertainties), axis=0)
+ # compute the EAvUC
+ eavuc = compute_eavuc(preds, labels, confs, unctns)
+ # compute ECE
+ accs, confs, num_Bm, conf_intervals = eval_calibration(preds, 1-unctns, labels, M=args.M)
+ ece = np.sum(np.abs(accs - confs) * num_Bm / np.sum(num_Bm))
+ return ece, eavuc, conf_intervals, accs
+
+
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser(description='MMAction2 test')
+ # model config
+ parser.add_argument('--ood_result', help='the result file of ood detection')
+ parser.add_argument('--M', type=int, default=15, help='The number of bins')
+ parser.add_argument('--ind_ncls', type=int, default=101, help='the number classes for in-distribution data')
+ parser.add_argument('--threshold', type=float, help='the threshold to decide if it is an OOD')
+ parser.add_argument('--save_prefix', help='the image file path of generated calibration figure')
+ parser.add_argument('--draw_diagram', action='store_true', help='if to draw reliability diagram.')
+ args = parser.parse_args()
+
+ results = np.load(args.ood_result, allow_pickle=True)
+ ind_uncertainties = results['ind_unctt'] # (N1,)
+ ood_uncertainties = results['ood_unctt'] # (N2,)
+ ind_results = results['ind_pred'] # (N1,)
+ ood_results = results['ood_pred'] # (N2,)
+ ind_labels = results['ind_label']
+ ood_labels = results['ood_label']
+ if 'ind_conf' not in results:
+ ind_confidences = 1 - ind_uncertainties
+ ood_confidences = 1 - ood_uncertainties
+ else:
+ ind_confidences = results['ind_conf']
+ ood_confidences = results['ood_conf']
+
+ # result path
+ result_path = os.path.dirname(args.save_prefix)
+ if not os.path.exists(result_path):
+ os.makedirs(result_path)
+
+ # Closed Set: (K class)
+ ece, eavuc, conf_intervals, accs = closedset_multicls(ind_results, ind_labels, ind_confidences, ind_uncertainties)
+ print('The Closed Set (K class) ECE=%.3lf, EAvUC=%.3lf'%(ece, eavuc))
+
+ # Open Set: K+1 class
+ ece, eavuc, conf_intervals, accs = openset_multicls(ind_results, ood_results, ind_labels, ood_labels, ind_confidences, ood_confidences, ind_uncertainties, ood_uncertainties)
+ print('The Open Set (K+1 class) ECE=%.3lf, EAvUC=%.3lf'%(ece, eavuc))
+
+ # Open Set: 2 class
+ ece, eavuc, conf_intervals, accs = openset_bincls(ind_results, ood_results, ind_labels, ood_labels, ind_confidences, ood_confidences, ind_uncertainties, ood_uncertainties)
+ print('The Open Set (2-class) ECE=%.3lf, EAvUC=%.3lf'%(ece, eavuc))
+
+ if args.draw_diagram:
+ # plot the ECE figure
+ fig, ax = plt.subplots(figsize=(4,4))
+ plt.rcParams["font.family"] = "Arial" # Times New Roman
+ fontsize = 15
+ plt.bar(conf_intervals, accs, width=1/args.M, linewidth=1, edgecolor='k', align='edge', label='Outputs')
+ plt.bar(conf_intervals, np.maximum(0, conf_intervals - accs), bottom=accs, color='y', width=1/args.M, linewidth=1, edgecolor='k', align='edge', label='Gap')
+ plt.text(0.1, 0.6, 'ECE=%.4f'%(ece), fontsize=fontsize)
+ add_identity(ax, color='r', ls='--')
+ plt.xlim(0, 1)
+ plt.ylim(0, 1)
+ plt.xticks(fontsize=fontsize)
+ plt.yticks(fontsize=fontsize)
+ plt.xlabel('confidence', fontsize=fontsize)
+ plt.ylabel('accuracy', fontsize=fontsize)
+ plt.legend(fontsize=fontsize)
+ ax.set_aspect('equal', 'box')
+ plt.tight_layout()
+ plt.savefig(args.save_prefix + '_ind.png')
+ plt.savefig(args.save_prefix + '_ind.pdf')
+
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/get_threshold.py b/Downstream/Open-Set-Action-Recognition/experiments/get_threshold.py
new file mode 100644
index 0000000..2138dc6
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/get_threshold.py
@@ -0,0 +1,183 @@
+import argparse
+import os
+import os.path as osp
+import torch
+import mmcv
+from mmaction.apis import init_recognizer
+from mmcv.parallel import collate, scatter
+from operator import itemgetter
+from mmaction.datasets.pipelines import Compose
+from mmaction.datasets import build_dataloader, build_dataset
+from mmcv.parallel import MMDataParallel
+import numpy as np
+from scipy.special import xlogy
+from tqdm import tqdm
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='MMAction2 test')
+ # model and data config
+ parser.add_argument('--config', help='test config file path')
+ parser.add_argument('--checkpoint', help='checkpoint file/url')
+ parser.add_argument('--uncertainty', default='BALD', choices=['BALD', 'Entropy', 'EDL'], help='the uncertainty estimation method')
+ parser.add_argument('--train_data', help='the split file of in-distribution training data')
+ parser.add_argument('--forward_pass', type=int, default=10, help='the number of forward passes')
+ parser.add_argument('--batch_size', type=int, default=8, help='the testing batch size')
+ # env config
+ parser.add_argument('--device', type=str, default='cuda:0', help='CPU/CUDA device option')
+ parser.add_argument('--result_prefix', help='result file prefix')
+ args = parser.parse_args()
+ return args
+
+def apply_dropout(m):
+ if type(m) == torch.nn.Dropout:
+ m.train()
+
+def update_seed(seed):
+ torch.manual_seed(seed)
+ torch.cuda.manual_seed(seed)
+
+def compute_uncertainty(predictions, method='BALD'):
+ """Compute the entropy
+ scores: (B x C x T)
+ """
+ expected_p = np.mean(predictions, axis=-1) # mean of all forward passes (C,)
+ entropy_expected_p = - np.sum(xlogy(expected_p, expected_p), axis=1) # the entropy of expect_p (across classes)
+ if method == 'Entropy':
+ uncertain_score = entropy_expected_p
+ elif method == 'BALD':
+ expected_entropy = - np.mean(np.sum(xlogy(predictions, predictions), axis=1), axis=-1) # mean of entropies (across classes), (scalar)
+ uncertain_score = entropy_expected_p - expected_entropy
+ else:
+ raise NotImplementedError
+ if not np.all(np.isfinite(uncertain_score)):
+ uncertain_score[~np.isfinite] = 9999
+ return uncertain_score
+
+def run_stochastic_inference(model, data_loader, forward_passes):
+ model = MMDataParallel(model, device_ids=[0])
+ all_uncertainties = []
+ prog_bar = mmcv.ProgressBar(len(data_loader.dataset))
+ for i, data in enumerate(data_loader):
+ all_scores = []
+ with torch.no_grad():
+ for n in range(forward_passes):
+ # set new random seed
+ update_seed(n * 1234)
+ scores = model(return_loss=False, **data)
+ # gather results
+ all_scores.append(np.expand_dims(scores, axis=-1))
+ all_scores = np.concatenate(all_scores, axis=-1) # (B, C, T)
+ # compute the uncertainty
+ uncertainty = compute_uncertainty(all_scores, method=args.uncertainty)
+ all_uncertainties.append(uncertainty)
+
+ # use the first key as main key to calculate the batch size
+ batch_size = len(next(iter(data.values())))
+ for _ in range(batch_size):
+ prog_bar.update()
+ all_uncertainties = np.concatenate(all_uncertainties, axis=0)
+
+ return all_uncertainties
+
+
+def run_evidence_inference(model, data_loader, evidence):
+
+ # get the evidence function
+ if evidence == 'relu':
+ from mmaction.models.losses.edl_loss import relu_evidence as get_evidence
+ elif evidence == 'exp':
+ from mmaction.models.losses.edl_loss import exp_evidence as get_evidence
+ elif evidence == 'softplus':
+ from mmaction.models.losses.edl_loss import softplus_evidence as get_evidence
+ else:
+ raise NotImplementedError
+ num_classes = model.cls_head.num_classes
+
+ model = MMDataParallel(model, device_ids=[0])
+ all_uncertainties = []
+ prog_bar = mmcv.ProgressBar(len(data_loader.dataset))
+ for i, data in enumerate(data_loader):
+ with torch.no_grad():
+ output = model(return_loss=False, **data)
+ evidence = get_evidence(torch.from_numpy(output))
+ alpha = evidence + 1
+ uncertainty = num_classes / torch.sum(alpha, dim=1)
+ scores = alpha / torch.sum(alpha, dim=1, keepdim=True)
+ all_uncertainties.append(uncertainty.numpy())
+
+ # use the first key as main key to calculate the batch size
+ batch_size = len(next(iter(data.values())))
+ for _ in range(batch_size):
+ prog_bar.update()
+ all_uncertainties = np.concatenate(all_uncertainties, axis=0)
+
+ return all_uncertainties
+
+
+def run_inference():
+
+ # build the recognizer from a config file and checkpoint file/url
+ model = init_recognizer(
+ args.config,
+ args.checkpoint,
+ device=device,
+ use_frames=False)
+ cfg = model.cfg
+
+ if not args.uncertainty == 'EDL':
+ # use dropout in testing stage
+ if 'dnn' in args.config:
+ model.apply(apply_dropout)
+ if 'bnn' in args.config:
+ model.test_cfg.npass = 1
+ # set cudnn benchmark
+ torch.backends.cudnn.benchmark = True
+ cfg.data.test.test_mode = True
+ cfg.data.videos_per_gpu = args.batch_size
+ evidence = cfg.get('evidence', 'exp')
+
+ # We use training data to obtain the threshold
+ cfg.data.test.ann_file = args.train_data
+ cfg.data.test.data_prefix = os.path.join(os.path.dirname(args.train_data), 'videos')
+ # build the dataloader
+ dataset = build_dataset(cfg.data.test, dict(test_mode=True))
+ dataloader_setting = dict(
+ videos_per_gpu=cfg.data.get('videos_per_gpu', 1),
+ workers_per_gpu=cfg.data.get('workers_per_gpu', 1),
+ dist=False,
+ shuffle=False,
+ pin_memory=False)
+ dataloader_setting = dict(dataloader_setting, **cfg.data.get('test_dataloader', {}))
+ data_loader = build_dataloader(dataset, **dataloader_setting)
+
+ # run inference
+ if not args.uncertainty == 'EDL':
+ all_uncertainties = run_stochastic_inference(model, data_loader, args.forward_pass)
+ else:
+ all_uncertainties = run_evidence_inference(model, data_loader, evidence)
+ return all_uncertainties
+
+
+if __name__ == '__main__':
+
+ args = parse_args()
+ # assign the desired device.
+ device = torch.device(args.device)
+
+ result_file = os.path.join(args.result_prefix + '_trainset_uncertainties.npz')
+ if not os.path.exists(result_file):
+ # run the inference on the entire training set (takes long time)
+ all_uncertainties = run_inference()
+ np.savez(result_file[:-4], uncertainty=all_uncertainties)
+ else:
+ result = np.load(result_file)
+ all_uncertainties = result['uncertainty']
+
+ # evaluation by macro-F1 within (C1 + 1) classes
+ uncertain_sort = np.sort(all_uncertainties)[::-1] # sort the uncertainties with descending order
+ N = all_uncertainties.shape[0]
+ topK = N - int(N * 0.95)
+ threshold = uncertain_sort[topK-1]
+
+ print('The model %s uncertainty threshold on UCF-101 train set: %lf'%(args.result_prefix.split('/')[-1], threshold))
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/get_threshold_dist.py b/Downstream/Open-Set-Action-Recognition/experiments/get_threshold_dist.py
new file mode 100644
index 0000000..1b18b9c
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/get_threshold_dist.py
@@ -0,0 +1,280 @@
+import argparse
+import os
+import os.path as osp
+import torch
+import mmcv
+from mmaction.apis import init_recognizer
+from mmcv.parallel import collate, scatter
+from operator import itemgetter
+from mmaction.datasets.pipelines import Compose
+from mmaction.datasets import build_dataloader, build_dataset
+from mmcv.parallel import MMDataParallel, MMDistributedDataParallel
+import numpy as np
+from scipy.special import xlogy
+from tqdm import tqdm
+import pdb
+from mmaction.apis import collect_results_cpu
+from mmcv.runner import get_dist_info, init_dist, load_checkpoint
+#from mmcv.runner import init_dist, set_random_seed
+from mmcv import Config, DictAction
+from mmaction.models import build_model
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='MMAction2 test')
+ # model and data config
+ parser.add_argument('--config', help='test config file path')
+ parser.add_argument('--checkpoint', help='checkpoint file/url')
+ parser.add_argument('--uncertainty', default='BALD', choices=['BALD', 'Entropy', 'EDL'], help='the uncertainty estimation method')
+ parser.add_argument('--train_data', help='the split file of in-distribution training data')
+ parser.add_argument('--forward_pass', type=int, default=10, help='the number of forward passes')
+ parser.add_argument('--batch_size', type=int, default=8, help='the testing batch size')
+ # env config
+ parser.add_argument('--device', type=str, default='cuda:0', help='CPU/CUDA device option')
+ parser.add_argument('--result_prefix', help='result file prefix')
+ parser.add_argument('--launcher', choices=['none', 'pytorch', 'slurm', 'mpi'], default='none', help='job launcher')
+ parser.add_argument('--local_rank', type=int, default=0)
+ parser.add_argument('--cfg-options', nargs='+', action=DictAction, default={},
+ help='override some settings in the used config, the key-value pair '
+ 'in xxx=yyy format will be merged into config file. For example, '
+ "'--cfg-options model.backbone.depth=18 model.backbone.with_cp=True'")
+ parser.add_argument('--fuse-conv-bn', action='store_true', help='Whether to fuse conv and bn, this will slightly increase''the inference speed')
+ args = parser.parse_args()
+ return args
+
+def apply_dropout(m):
+ if type(m) == torch.nn.Dropout:
+ m.train()
+
+def update_seed(seed):
+ torch.manual_seed(seed)
+ torch.cuda.manual_seed(seed)
+
+def compute_uncertainty(predictions, method='BALD'):
+ """Compute the entropy
+ scores: (B x C x T)
+ """
+ expected_p = np.mean(predictions, axis=-1) # mean of all forward passes (C,)
+ entropy_expected_p = - np.sum(xlogy(expected_p, expected_p), axis=1) # the entropy of expect_p (across classes)
+ if method == 'Entropy':
+ uncertain_score = entropy_expected_p
+ elif method == 'BALD':
+ expected_entropy = - np.mean(np.sum(xlogy(predictions, predictions), axis=1), axis=-1) # mean of entropies (across classes), (scalar)
+ uncertain_score = entropy_expected_p - expected_entropy
+ else:
+ raise NotImplementedError
+ if not np.all(np.isfinite(uncertain_score)):
+ uncertain_score[~np.isfinite] = 9999
+ return uncertain_score
+
+def run_stochastic_inference(model, data_loader, forward_passes):
+ # model = MMDataParallel(model, device_ids=[0])
+ model.eval()
+ all_uncertainties = []
+ # prog_bar = mmcv.ProgressBar(len(data_loader.dataset))
+ rank, world_size = get_dist_info()
+ if rank == 0:
+ prog_bar = mmcv.ProgressBar(len(data_loader.dataset))
+
+ for i, data in enumerate(data_loader):
+ all_scores = []
+ with torch.no_grad():
+ for n in range(forward_passes):
+ # set new random seed
+ update_seed(n * 1234)
+ # original code 69-76
+ # scores = model(return_loss=False, **data)
+ # # gather results
+ # all_scores.append(np.expand_dims(scores, axis=-1))
+ # all_scores = np.concatenate(all_scores, axis=-1) # (B, C, T)
+ # # compute the uncertainty
+ # uncertainty = compute_uncertainty(all_scores, method=args.uncertainty)
+ # all_uncertainties.append(uncertainty)
+ #---vae---------------------------------
+ # pdb.set_trace()
+ scores, recon = model(return_loss=False, **data)
+ uncertainty = recon
+ all_scores.append(np.expand_dims(scores, axis=-1))
+ all_scores = np.concatenate(all_scores, axis=-1) # (B, C, T)
+ all_uncertainties.append(uncertainty)
+ #-----------------------------------------------
+
+ # use the first key as main key to calculate the batch size
+ # batch_size = len(next(iter(data.values())))
+ # for _ in range(batch_size):
+ # prog_bar.update()
+
+ if rank == 0:
+ # use the first key as main key to calculate the batch size
+ batch_size = len(next(iter(data.values())))
+ for _ in range(batch_size * world_size):
+ prog_bar.update()
+
+ all_uncertainties = collect_results_cpu(all_uncertainties, len(data_loader.dataset), tmpdir=None)
+ rank, _ = get_dist_info()
+ if rank == 0:
+ all_uncertainties = np.concatenate(all_uncertainties, axis=0)
+
+ return all_uncertainties
+
+def run_evidence_inference(model, data_loader, evidence):
+
+ # get the evidence function
+ if evidence == 'relu':
+ from mmaction.models.losses.edl_loss import relu_evidence as get_evidence
+ elif evidence == 'exp':
+ from mmaction.models.losses.edl_loss import exp_evidence as get_evidence
+ elif evidence == 'softplus':
+ from mmaction.models.losses.edl_loss import softplus_evidence as get_evidence
+ else:
+ raise NotImplementedError
+ num_classes = 101
+ model.eval()
+
+ # model = MMDataParallel(model, device_ids=[0])
+ all_uncertainties = []
+ # prog_bar = mmcv.ProgressBar(len(data_loader.dataset))
+ rank, world_size = get_dist_info()
+ if rank == 0:
+ prog_bar = mmcv.ProgressBar(len(data_loader.dataset))
+
+ for i, data in enumerate(data_loader):
+ with torch.no_grad():
+ # output = model(return_loss=False, **data)
+ output = model(return_loss=False, **data)
+ evidence = get_evidence(torch.from_numpy(output))
+ alpha = evidence + 1
+ uncertainty = num_classes / torch.sum(alpha, dim=1)
+ scores = alpha / torch.sum(alpha, dim=1, keepdim=True)
+ all_uncertainties.append(uncertainty.numpy())
+ #---vae---------------------------------
+ # output, recon = model(return_loss=False, **data)
+ # uncertainty = recon
+ # all_uncertainties.append(uncertainty)
+ #------------------------------------------
+ # pdb.set_trace()
+ # use the first key as main key to calculate the batch size
+ # batch_size = len(next(iter(data.values())))
+ # for _ in range(batch_size):
+ # prog_bar.update()
+ if rank == 0:
+ # use the first key as main key to calculate the batch size
+ batch_size = len(next(iter(data.values())))
+ for _ in range(batch_size * world_size):
+ prog_bar.update()
+ all_uncertainties = collect_results_cpu(all_uncertainties, len(data_loader.dataset), tmpdir=None)
+ rank, _ = get_dist_info()
+ if rank == 0:
+ all_uncertainties = np.concatenate(all_uncertainties, axis=0)
+
+ return all_uncertainties
+
+
+def run_inference():
+
+ # build the recognizer from a config file and checkpoint file/url
+ # model = init_recognizer(
+ # args.config,
+ # args.checkpoint,
+ # device=device,
+ # use_frames=False)
+ # cfg = model.cfg
+
+ model = build_model(cfg.model, train_cfg=None, test_cfg=cfg.test_cfg)
+ fp16_cfg = cfg.get('fp16', None)
+ if fp16_cfg is not None:
+ wrap_fp16_model(model)
+ load_checkpoint(model, args.checkpoint, map_location='cpu')
+ print('cpk is loaded:', args.checkpoint)
+
+ if args.fuse_conv_bn:
+ model = fuse_conv_bn(model)
+
+
+ model = MMDistributedDataParallel(
+ model.cuda(),
+ device_ids=[torch.cuda.current_device()],
+ broadcast_buffers=False)
+
+ # if not args.uncertainty == 'EDL':
+ # # use dropout in testing stage
+ # if 'dnn' in args.config:
+ # model.apply(apply_dropout)
+ # if 'bnn' in args.config:
+ # model.test_cfg.npass = 1
+
+ # set cudnn benchmark
+ torch.backends.cudnn.benchmark = True
+ cfg.data.test.test_mode = True
+ cfg.data.videos_per_gpu = args.batch_size
+ evidence = cfg.get('evidence', 'exp')
+
+ # We use training data to obtain the threshold
+ #pdb.set_trace()
+ cfg.data.test.ann_file = args.train_data
+ cfg.data.test.data_prefix = os.path.join(os.path.dirname(args.train_data), 'videos')
+ # build the dataloader
+ dataset = build_dataset(cfg.data.test, dict(test_mode=True))
+ dataloader_setting = dict(
+ videos_per_gpu=cfg.data.get('videos_per_gpu', 1),
+ workers_per_gpu=cfg.data.get('workers_per_gpu', 1),
+ dist=distributed,
+ shuffle=False,
+ pin_memory=False)
+ dataloader_setting = dict(dataloader_setting, **cfg.data.get('test_dataloader', {}))
+ #pdb.set_trace()
+ data_loader = build_dataloader(dataset, **dataloader_setting)
+
+ # run inference
+ if not args.uncertainty == 'EDL':
+ all_uncertainties = run_stochastic_inference(model, data_loader, args.forward_pass)
+ else:
+ all_uncertainties = run_evidence_inference(model, data_loader, evidence)
+ return all_uncertainties
+
+
+if __name__ == '__main__':
+
+ # args = parse_args()
+ # # assign the desired device.
+ # device = torch.device(args.device)
+
+ # build the recognizer from a config file and checkpoint file/url
+ args = parse_args()
+ cfg = Config.fromfile(args.config)
+ cfg.merge_from_dict(args.cfg_options)
+
+ # init distributed env first, since logger depends on the dist info.
+ if args.launcher == 'none':
+ distributed = False
+ else:
+ distributed = True
+ init_dist(args.launcher, **cfg.dist_params)
+
+ # pdb.set_trace()
+ result_file = os.path.join(args.result_prefix + '_trainset_uncertainties.npz')
+ if not os.path.exists(result_file):
+ # run the inference on the entire training set (takes long time)
+ #pdb.set_trace()
+ all_uncertainties = run_inference()
+ np.savez(result_file[:-4], uncertainty=all_uncertainties)
+ else:
+ result = np.load(result_file)
+ all_uncertainties = result['uncertainty']
+
+ # evaluation by macro-F1 within (C1 + 1) classes
+ # pdb.set_trace()
+ rank, _ = get_dist_info()
+ if rank == 0:
+ # uncertain_sort = np.sort(all_uncertainties)[::-1] # sort the uncertainties with descending order
+ uncertain_sort = np.sort(all_uncertainties.squeeze())
+ #uncertain_sort = (uncertain_sort - np.min(uncertain_sort)) / (np.max(uncertain_sort) - np.min(uncertain_sort)) # normalize
+
+ N = all_uncertainties.shape[0]
+ topK = N - int(N * 0.95)
+ print(uncertain_sort)
+ print(uncertain_sort.min())
+ print(uncertain_sort.max())
+ print(len(uncertain_sort))
+ threshold = uncertain_sort[topK-1]
+
+ print('The model %s uncertainty threshold on UCF-101 train set: %lf'%(args.result_prefix.split('/')[-1], threshold))
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/baseline_i3d_openmax.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/baseline_i3d_openmax.sh
new file mode 100644
index 0000000..c2118c9
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/baseline_i3d_openmax.sh
@@ -0,0 +1,40 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+DEVICE=$1
+OOD_DATASET=$2
+IND_DATA='data/ucf101/ucf101_val_split_1_videos.txt'
+
+case ${OOD_DATASET} in
+ HMDB)
+ # run ood detection on hmdb-51 validation set
+ OOD_DATA='data/hmdb51/hmdb51_val_split_1_videos.txt'
+ NUM_CLASSES=51
+ ;;
+ MiT)
+ # run ood detection on mit-v2 validation set
+ OOD_DATA='data/mit/mit_val_list_videos.txt'
+ NUM_CLASSES=305
+ ;;
+ *)
+ echo "Dataset not supported: "${OOD_DATASET}
+ exit
+ ;;
+esac
+RESULT_DIR='experiments/i3d/results'
+
+CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/baseline_openmax.py \
+ --config configs/recognition/i3d/inference_i3d_dnn.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_dnn/latest.pth \
+ --cache_mav_dist experiments/i3d/results_baselines/openmax/ucf101_mav_dist \
+ --ind_data ${IND_DATA} \
+ --ood_data ${OOD_DATA} \
+ --ood_ncls ${NUM_CLASSES} \
+ --result_prefix experiments/i3d/results_baselines/openmax/I3D_OpenMax_${OOD_DATASET}
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/baseline_i3d_rpl.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/baseline_i3d_rpl.sh
new file mode 100644
index 0000000..554bc6f
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/baseline_i3d_rpl.sh
@@ -0,0 +1,40 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+DEVICE=$1
+OOD_DATASET=$2
+IND_DATA='data/ucf101/ucf101_val_split_1_videos.txt'
+
+case ${OOD_DATASET} in
+ HMDB)
+ # run ood detection on hmdb-51 validation set
+ OOD_DATA='data/hmdb51/hmdb51_val_split_1_videos.txt'
+ NUM_CLASSES=51
+ ;;
+ MiT)
+ # run ood detection on mit-v2 validation set
+ OOD_DATA='data/mit/mit_val_list_videos.txt'
+ NUM_CLASSES=305
+ ;;
+ *)
+ echo "Dataset not supported: "${OOD_DATASET}
+ exit
+ ;;
+esac
+
+CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/baseline_rpl.py \
+ --config configs/recognition/i3d/inference_i3d_rpl.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_rpl/latest.pth \
+ --train_data data/ucf101/ucf101_train_split_1_videos.txt \
+ --ind_data ${IND_DATA} \
+ --ood_data ${OOD_DATA} \
+ --ood_ncls ${NUM_CLASSES} \
+ --ood_dataname ${OOD_DATASET} \
+ --result_prefix experiments/i3d/results_baselines/rpl/RPL
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/baseline_i3d_softmax.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/baseline_i3d_softmax.sh
new file mode 100644
index 0000000..f88f6b4
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/baseline_i3d_softmax.sh
@@ -0,0 +1,33 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+DEVICE=$1
+OOD_DATASET=$2
+
+case ${OOD_DATASET} in
+ HMDB)
+ NUM_CLASSES=51
+ ;;
+ MiT)
+ NUM_CLASSES=305
+ ;;
+ *)
+ echo "Dataset not supported: "${OOD_DATASET}
+ exit
+ ;;
+esac
+RESULT_DIR='experiments/i3d/results'
+
+CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/baseline_softmax.py \
+ --config configs/recognition/i3d/inference_i3d_dnn.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_dnn/latest.pth \
+ --batch_size 4 \
+ --ood_ncls ${NUM_CLASSES} \
+ --result_prefix experiments/i3d/results_baselines/softmax/I3D_SoftMax_${OOD_DATASET}
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_bnn_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_bnn_ucf101.sh
new file mode 100644
index 0000000..827fd26
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_bnn_ucf101.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+CUDA_VISIBLE_DEVICES=$1 python tools/test.py configs/recognition/i3d/finetune_ucf101_i3d_bnn.py \
+ work_dirs/i3d/finetune_ucf101_i3d_bnn/latest.pth \
+ --out work_dirs/i3d/test_ucf101_i3d_bnn.pkl \
+ --eval top_k_accuracy mean_class_accuracy
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_dnn_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_dnn_ucf101.sh
new file mode 100644
index 0000000..61d3542
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_dnn_ucf101.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+CUDA_VISIBLE_DEVICES=$1 python tools/test.py configs/recognition/i3d/finetune_ucf101_i3d_dnn.py \
+ work_dirs/i3d/finetune_ucf101_i3d_dnn/latest.pth \
+ --out work_dirs/i3d/test_ucf101_i3d_dnn.pkl \
+ --eval top_k_accuracy mean_class_accuracy
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_edlnokl_avuc_debias_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_edlnokl_avuc_debias_ucf101.sh
new file mode 100644
index 0000000..ba296d8
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_edlnokl_avuc_debias_ucf101.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+CUDA_VISIBLE_DEVICES=$1 python tools/test.py configs/recognition/i3d/finetune_ucf101_i3d_edlnokl_avuc_debias.py \
+ work_dirs/i3d/finetune_ucf101_i3d_edlnokl_avuc_debias/latest.pth \
+ --out work_dirs/i3d/test_ucf101_i3d_edlnokl_avuc_debias.pkl \
+ --eval top_k_accuracy mean_class_accuracy
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_edlnokl_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_edlnokl_ucf101.sh
new file mode 100644
index 0000000..7ceed5f
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_edlnokl_ucf101.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+CUDA_VISIBLE_DEVICES=$1 python tools/test.py configs/recognition/i3d/finetune_ucf101_i3d_edlnokl.py \
+ work_dirs/i3d/finetune_ucf101_i3d_edlnokl/latest.pth \
+ --out work_dirs/i3d/test_ucf101_i3d_edlnokl.pkl \
+ --eval top_k_accuracy mean_class_accuracy
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_rpl_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_rpl_ucf101.sh
new file mode 100644
index 0000000..ac86139
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/evaluate_i3d_rpl_ucf101.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+CUDA_VISIBLE_DEVICES=$1 python tools/test.py configs/recognition/i3d/finetune_ucf101_i3d_rpl.py \
+ work_dirs/i3d/finetune_ucf101_i3d_rpl/latest.pth \
+ --out work_dirs/i3d/test_ucf101_i3d_rpl.pkl \
+ --eval top_k_accuracy mean_class_accuracy
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_bnn_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_bnn_ucf101.sh
new file mode 100644
index 0000000..63b5353
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_bnn_ucf101.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+CUDA_VISIBLE_DEVICES=$1 python tools/train.py configs/recognition/i3d/finetune_ucf101_i3d_bnn.py \
+ --work-dir work_dirs/i3d/finetune_ucf101_i3d_bnn \
+ --validate \
+ --seed 0 \
+ --deterministic \
+ --gpu-ids 0
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_dnn_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_dnn_ucf101.sh
new file mode 100644
index 0000000..74617b9
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_dnn_ucf101.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+CUDA_VISIBLE_DEVICES=$1 python tools/train.py configs/recognition/i3d/finetune_ucf101_i3d_dnn.py \
+ --work-dir work_dirs/i3d/finetune_ucf101_i3d_dnn \
+ --validate \
+ --seed 0 \
+ --deterministic \
+ --gpu-ids 0
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_edlnokl_avuc_debias_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_edlnokl_avuc_debias_ucf101.sh
new file mode 100644
index 0000000..00af2d0
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_edlnokl_avuc_debias_ucf101.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+# --validate
+CUDA_VISIBLE_DEVICES=$1 python tools/train.py configs/recognition/i3d/finetune_ucf101_i3d_edlnokl_avuc_debias.py \
+ --work-dir work_dirs/i3d/finetune_ucf101_i3d_edlnokl_avuc_debias \
+ --validate \
+ --seed 0 \
+ --deterministic \
+ --gpu-ids 0
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_edlnokl_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_edlnokl_ucf101.sh
new file mode 100644
index 0000000..9494db9
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_edlnokl_ucf101.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+# --validate
+CUDA_VISIBLE_DEVICES=$1 python tools/train.py configs/recognition/i3d/finetune_ucf101_i3d_edlnokl.py \
+ --work-dir work_dirs/i3d/finetune_ucf101_i3d_edlnokl \
+ --validate \
+ --seed 0 \
+ --deterministic \
+ --gpu-ids 0
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_rpl_ucf101.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_rpl_ucf101.sh
new file mode 100644
index 0000000..7a16c6b
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/finetune_i3d_rpl_ucf101.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+CUDA_VISIBLE_DEVICES=$1 python tools/train.py configs/recognition/i3d/finetune_ucf101_i3d_rpl.py \
+ --work-dir work_dirs/i3d/finetune_ucf101_i3d_rpl \
+ --validate \
+ --seed 0 \
+ --deterministic \
+ --gpu-ids 0
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_draw_confmat.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_draw_confmat.sh
new file mode 100644
index 0000000..f938880
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_draw_confmat.sh
@@ -0,0 +1,20 @@
+#!/bin/bash
+
+export CUDA_HOME='/usr/local/cuda'
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+OOD_DATA=$1 # HMDB or MiT
+RESULT_PATH="experiments/i3d/results"
+
+# Confusion Matrix comparison
+python experiments/draw_confusion_matrix.py \
+ --ood_result ${RESULT_PATH}/I3D_EDLNoKLAvUCDebias_EDL_${OOD_DATA}_result.npz \
+ --uncertain_thresh 0.004550 \
+ --save_file ${RESULT_PATH}/../results_confmat/I3D_DEAR_${OOD_DATA}_ConfMat.png
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_get_threshold.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_get_threshold.sh
new file mode 100644
index 0000000..d2ffd22
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_get_threshold.sh
@@ -0,0 +1,65 @@
+#!/bin/bash
+
+export CUDA_HOME='/usr/local/cuda'
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+DEVICE=$1
+MODEL=$2
+BATCHSIZE=$3
+TRAIN_DATA='data/ucf101/ucf101_train_split_1_videos.txt'
+RESULT_DIR='experiments/i3d/results'
+
+case ${MODEL} in
+ dnn)
+ # get the BALD threshold for i3d model trained on UCF-101
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/get_threshold.py \
+ --config configs/recognition/i3d/inference_i3d_dnn.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_dnn/latest.pth \
+ --train_data ${TRAIN_DATA} \
+ --batch_size ${BATCHSIZE} \
+ --uncertainty BALD \
+ --result_prefix ${RESULT_DIR}/I3D_DNN_BALD
+ ;;
+ bnn)
+ # get the BALD threshold for I3D_BNN model trained on UCF-101
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/get_threshold.py \
+ --config configs/recognition/i3d/inference_i3d_bnn.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_bnn/latest.pth \
+ --train_data ${TRAIN_DATA} \
+ --batch_size ${BATCHSIZE} \
+ --uncertainty BALD \
+ --result_prefix ${RESULT_DIR}/I3D_BNN_BALD
+ ;;
+ edlnokl)
+ # get the EDL threshold for I3D_EDL model trained on UCF-101
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/get_threshold.py \
+ --config configs/recognition/i3d/inference_i3d_enn.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_edlnokl/latest.pth \
+ --train_data ${TRAIN_DATA} \
+ --batch_size ${BATCHSIZE} \
+ --uncertainty EDL \
+ --result_prefix ${RESULT_DIR}/I3D_EDLNoKL_EDL
+ ;;
+ edlnokl_avuc_debias)
+ # get the EDL threshold for I3D_EDL_AvUC_Debias model trained on UCF-101
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/get_threshold.py \
+ --config configs/recognition/i3d/inference_i3d_enn.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_edlnokl_avuc_debias/latest.pth \
+ --train_data ${TRAIN_DATA} \
+ --batch_size ${BATCHSIZE} \
+ --uncertainty EDL \
+ --result_prefix ${RESULT_DIR}/I3D_EDLNoKLAvUCDebias_EDL
+ ;;
+ *)
+ echo "Invalid model: "${MODEL}
+ exit
+ ;;
+esac
+
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_ood_detection.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_ood_detection.sh
new file mode 100644
index 0000000..37e3196
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_ood_detection.sh
@@ -0,0 +1,80 @@
+#!/bin/bash
+
+export CUDA_HOME='/usr/local/cuda'
+
+pwd_dir=$pwd
+cd ../../
+# DEVICE=$1
+OOD_DATASET=$1
+MODEL=$2
+GPUS=$3
+PORT=${PORT:-29500}
+IND_DATA='data/ucf101/ucf101_val_split_1_videos.txt'
+
+case ${OOD_DATASET} in
+ HMDB)
+ # run ood detection on hmdb-51 validation set
+ OOD_DATA='data/hmdb51/hmdb51_val_split_1_videos.txt'
+ ;;
+ MiT)
+ # run ood detection on hmdb-51 validation set
+ OOD_DATA='data/mit/mit_val_list_videos.txt'
+ ;;
+ *)
+ echo "Dataset not supported: "${OOD_DATASET}
+ exit
+ ;;
+esac
+RESULT_DIR='experiments/i3d/results'
+
+case ${MODEL} in
+ dnn)
+ # DNN with Dropout model
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/ood_detection.py \
+ --config configs/recognition/i3d/inference_i3d_dnn.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_dnn/latest.pth \
+ --ind_data ${IND_DATA} \
+ --ood_data ${OOD_DATA} \
+ --uncertainty BALD \
+ --result_prefix ${RESULT_DIR}/I3D_DNN_BALD_${OOD_DATASET}
+ ;;
+ bnn)
+ # Evidential Deep Learning (without KL divergence loss term)
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/ood_detection.py \
+ --config configs/recognition/i3d/inference_i3d_bnn.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_bnn/latest.pth \
+ --ind_data ${IND_DATA} \
+ --ood_data ${OOD_DATA} \
+ --uncertainty BALD \
+ --result_prefix ${RESULT_DIR}/I3D_BNN_BALD_${OOD_DATASET}
+ ;;
+ edlnokl)
+ # Evidential Deep Learning (without KL divergence loss term)
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/ood_detection.py \
+ --config configs/recognition/i3d/inference_i3d_enn.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_edlnokl/latest.pth \
+ --ind_data ${IND_DATA} \
+ --ood_data ${OOD_DATA} \
+ --uncertainty EDL \
+ --result_prefix ${RESULT_DIR}/I3D_EDLNoKL_EDL_${OOD_DATASET}
+ ;;
+ edlnokl_avuc_debias)
+ # Evidential Deep Learning (without KL divergence loss term) with AvU Calibration and Debiasing
+ python -m torch.distributed.launch --nproc_per_node=$GPUS --master_port=$PORT experiments/ood_detection_dist.py \
+ --config configs/recognition/i3d/inference_i3d_enn.py \
+ --checkpoint work_dirs/i3d/finetune_ucf101_i3d_edlnokl_avuc_ced/latest.pth \
+ --ind_data ${IND_DATA} \
+ --ood_data ${OOD_DATA} \
+ --uncertainty EDL \
+ --result_prefix ${RESULT_DIR}/I3D_EDLNoKLAvUCCED_EDL_${OOD_DATASET} \
+ --launcher pytorch
+ ;;
+ *)
+ echo "Invalid model: "${MODEL}
+ exit
+ ;;
+esac
+
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_openness.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_openness.sh
new file mode 100644
index 0000000..54552ad
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_openness.sh
@@ -0,0 +1,37 @@
+#!/bin/bash
+
+export CUDA_HOME='/usr/local/cuda'
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+OOD_DATA=$1 # HMDB or MiT
+case ${OOD_DATA} in
+ HMDB)
+ NUM_CLASSES=51
+ ;;
+ MiT)
+ NUM_CLASSES=305
+ ;;
+ *)
+ echo "Invalid OOD Dataset: "${OOD_DATA}
+ exit
+ ;;
+esac
+
+# OOD Detection comparison
+python experiments/compare_openness.py \
+ --base_model i3d \
+ --baselines I3D_DNN_BALD I3D_BNN_BALD I3D_EDLNoKL_EDL I3D_EDLNoKLAvUCCED_EDL I3D_EDLNoKLAvUCDebias_EDL \
+ --thresholds 0.000433 0.000004 0.004547 0.004550 0.004550 \
+ --styles b k y c r\
+ --ood_data ${OOD_DATA} \
+ --ood_ncls ${NUM_CLASSES} \
+ --ind_ncls 101 \
+ --result_png F1_openness_compare_${OOD_DATA}.png
+
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_openness_new.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_openness_new.sh
new file mode 100644
index 0000000..87da539
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_openness_new.sh
@@ -0,0 +1,40 @@
+#!/bin/bash
+
+export CUDA_HOME='/usr/local/cuda'
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+OOD_DATA=$1 # HMDB or MiT
+RESULT_FILES=("results_baselines/openmax/I3D_OpenMax_${OOD_DATA}_result.npz" \ # openmax
+ "results/I3D_DNN_BALD_${OOD_DATA}_result.npz" \ # mc dropout
+ "results/I3D_BNN_BALD_${OOD_DATA}_result.npz" \ # bnn svi
+ "results_baselines/openmax/I3D_OpenMax_${OOD_DATA}_result.npz" \ # softmax
+ "results_baselines/rpl/I3D_RPL_${OOD_DATA}_result.npz" \ # rpl
+ "results/I3D_EDLNoKLAvUCDebias_EDL_${OOD_DATA}_result.npz") # dear (ours)
+THRESHOLDS=(-1 \
+ 0.000433 \
+ 0.000004 \
+ 0.996825 \
+ 0.995178 \
+ 0.004550)
+
+# OOD Detection comparison
+# The folders `results/` and `results_baselines` are in the `experiments/i3d/` folder.
+python experiments/compare_openness_new.py \
+ --base_model i3d \
+ --ood_data ${OOD_DATA} \
+ --baseline_results ${RESULT_FILES[@]}
+
+# OOD Detection comparison by using thresholds
+echo 'Results by using thresholds:'
+python experiments/compare_openness_new.py \
+ --base_model i3d \
+ --ood_data ${OOD_DATA} \
+ --thresholds ${THRESHOLDS[@]} \
+ --baseline_results ${RESULT_FILES[@]}
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_reliability_evaluation.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_reliability_evaluation.sh
new file mode 100644
index 0000000..85ef69d
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/run_reliability_evaluation.sh
@@ -0,0 +1,47 @@
+#!/bin/bash
+
+export CUDA_HOME='/usr/local/cuda'
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+OOD_DATASET=$1
+MODEL=$2
+RESULT_DIR='experiments/i3d/results'
+
+case ${MODEL} in
+ dnn)
+ # DNN with Dropout model
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/evaluate_calibration.py \
+ --ood_result ${RESULT_DIR}/I3D_DNN_BALD_${OOD_DATASET}_result.npz \
+ --save_prefix ${RESULT_DIR}/../results_reliability/I3D_DNN_BALD_${OOD_DATASET}_reliability
+ ;;
+ bnn)
+ # Evidential Deep Learning (without KL divergence loss term)
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/evaluate_calibration.py \
+ --ood_result ${RESULT_DIR}/I3D_BNN_BALD_${OOD_DATASET}_result.npz \
+ --save_prefix ${RESULT_DIR}/../results_reliability/I3D_BNN_BALD_${OOD_DATASET}_reliability
+ ;;
+ edlnokl)
+ # Evidential Deep Learning (without KL divergence loss term)
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/evaluate_calibration.py \
+ --ood_result ${RESULT_DIR}/I3D_EDLNoKL_EDL_${OOD_DATASET}_result.npz \
+ --save_prefix ${RESULT_DIR}/../results_reliability/I3D_EDLNoKL_EDL_${OOD_DATASET}_reliability
+ ;;
+ edlnokl_avuc_debias)
+ # Evidential Deep Learning (without KL divergence loss term) with AvU Calibration and Debiasing
+ CUDA_VISIBLE_DEVICES=${DEVICE} python experiments/evaluate_calibration.py \
+ --ood_result ${RESULT_DIR}/I3D_EDLNoKLAvUCCED_EDL_${OOD_DATASET}_result.npz \
+ --save_prefix ${RESULT_DIR}/../results_reliability/I3D_EDLNoKLAvUCCED_EDL_${OOD_DATASET}_reliability
+ ;;
+ *)
+ echo "Invalid model: "${MODEL}
+ exit
+ ;;
+esac
+
+
+cd $pwd_dir
+echo "Experiments finished!"
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/train_i3d_DEAR_kinetics10.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/train_i3d_DEAR_kinetics10.sh
new file mode 100644
index 0000000..95eaaa0
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/train_i3d_DEAR_kinetics10.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+# --validate
+CUDA_VISIBLE_DEVICES=$1 python tools/train.py configs/recognition/i3d/train_kinetics10_i3d_DEAR.py \
+ --work-dir work_dirs/i3d/train_kinetics10_i3d_DEAR \
+ --validate \
+ --seed 0 \
+ --deterministic \
+ --gpu-ids 0
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/i3d/train_i3d_DEAR_noDebias_kinetics10.sh b/Downstream/Open-Set-Action-Recognition/experiments/i3d/train_i3d_DEAR_noDebias_kinetics10.sh
new file mode 100644
index 0000000..caa6748
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/i3d/train_i3d_DEAR_noDebias_kinetics10.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+pwd_dir=$pwd
+cd ../../
+
+source activate mmaction
+
+# --validate
+CUDA_VISIBLE_DEVICES=$1 python tools/train.py configs/recognition/i3d/train_kinetics10_i3d_DEAR_noDebias.py \
+ --work-dir work_dirs/i3d/train_kinetics10_i3d_DEAR_noDebias \
+ --validate \
+ --seed 0 \
+ --deterministic \
+ --gpu-ids 0
+
+cd $pwd_dir
+echo "Experiments finished!"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/libMR/COPYRIGHT_Libmr.txt b/Downstream/Open-Set-Action-Recognition/experiments/libMR/COPYRIGHT_Libmr.txt
new file mode 100644
index 0000000..084f881
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/libMR/COPYRIGHT_Libmr.txt
@@ -0,0 +1,98 @@
+SOURCE CODE LICENSE AGREEMENT
+PREAMBLE
+
+This SOFTWARE implements concepts of statistical Meta-recognition for which Securics/Univ. of Colorado have a pending patent (CU TTO File CU2338C). Securics and University of Colorado, hereafter The Owners, have joint interest in the invention and the software. Securics currently holds the excusive license commercial to both the patent and the code. Securcs, hereafter the Licensor, is offering a non-exclusive right to use for non-commecial use.
+This license agreement allows you to use the source code for personal or non profit purposes. This includes any use that does not involve making money, and does not include uses like:
+• deploying the software for use by a for-profit organization
+• providing a service to a paying customer
+For-profit companies may not use this source code. If you work for a for-profit company, you may only use this software as an individual, for your personal use.
+
+This is a right to use license. It does not not include the right to redistribute copies. Non-profit users can only use the copies obtained from authroized sourcehs which include: securics.com, metarecognition.com or vast.uccs.edu.
+
+This license agreement also allows you to create derivative products for your own use, but does not permit re-distribute
+of modified code in any form. You may choose to destribute patch files, which can be applied to officially distributed code. Any the derivative products, must be distributed under the same conditions as specified in this agreement unless a separate commercial license is obtained from Securics Inc or its designates.
+
+As a condition of using this source code, you agree not to assert any patents or copyrights against the owners or any of the Owners’ licensees for use of derivative products. Any derivative products must include a copy of license and instructions for accessing the orignal source. You must also include attribution to the authors in any publication that results from the use of this code or data derived from the code. Any papers/research/report based on results that uses this software must cite:
+
+@article{Scheirer_2011_TPAMI,
+author = {Walter J. Scheirer and Anderson Rocha and Ross Michaels and Terrance E. Boult},
+title = {Meta-Recognition: The Theory and Practice of Recognition Score Analysis},
+journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)},
+volume = {33},
+issue = {8},
+pages = {1689–1695},
+year = {2011}
+}
+Parts of this technology are subject to SBIR data rights and as described in DFARS 252.227-7018 (June 1995) SBIR Data Rights which apply to Contract Number: N00014-11-C-0243 and STTR N00014-07-M-0421 to Securics Inc, 1870 Austin Bluffs Parkway, Colorado Springs, CO 80918
+
+The Government’s rights to use, modify, reproduce, release, perform, display, or disclose technical data or computer software marked with this legend are restricted during the period shown as provided in paragraph (b)(4) of the Rights in Noncommercial Technical Data and Computer Software-Small Business Innovative Research (SBIR) Program clause contained in the above identified contract. Expiration of SBIR Data Rights: Expires four years after completion of the above cited project work for this or any other follow-on SBIR contract, whichever is later.
+
+No restrictions on government use apply after the expiration date shown above. Any reproduction of technical data, computer software, or portions thereof marked with this legend must also reproduce the markings.
+
+This license includes other conditions that should be read carefully. This SOFWARE usage agreement (the “Agreement”) applies to the libMR and is between YOU and the Licensor
+
+1. DEFINITIONS
+
+“Software” means all or any portion of the human-readable source code files of the software programs including without limitation, associated flow charts, algorithms, comments and other written instructions and technical documentation, and all corrections, updates, and new versions incorporated into such programs.
+
+“Derivative Work” means a work based upon the Software, such as a revision, modification, translation, abridgement, condensation, expansion, collection, compilation, or any other form in which the Software may be recast, transformed, adapted, or distributed as a part of a larger work and which, if prepared without proper authorization would constitute a copyright infringement. If identifiable sections of that work are not derived from the Software, and can be reasonably considered independent and separate works in themselves, then they are not considered Derivative Work.
+
+“Personal Use” means use of Software and/or Derivative Work by an individual solely for his or her personal, private and non-commercial use. An individual’s use in his or her capacity as an officer, employee, member, independent contractor or agent of a corporation, business or organization does not qualify as Personal Use.
+
+“You” or “Your” means an individual or a legal entity exercising rights under this License. For legal entities, “You” or “Your” includes any non-profit entity which controls, is controlled by, or is under common control with, You, where “control” means (a) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (b) ownership of fifty percent (50%) or more of the beneficial ownership of such entity.
+
+2. GRANT OF LICENSE:
+
+WHEREAS, the Licensor, desires to aid the academic and non-commercial research community and raise awareness of the PATENTED INVENTION and thereby agrees to grant a limited copyright license to the SOFTWARE for research and non-commercial purposes only, with the Owners retaining all ownership rights in the PATENTED INVENTION and the SOFTWARE;
+
+THEREFORE:
+The Licensor grants, and You accept, a personal, nonexclusive, nontransferable license:
+
+a) to use Software, at no charge, in accordance with the terms herein, solely for (i) Personal Use, or (ii) academic or non-commercial research, development and deployment; and
+
+b) to develop Derivative Works that may be used solely for (i) Personal Use or (ii) academic or non-commercial research, development and deployment; and
+
+c) to copy, distribute and sublicense Software and Derivative Works solely in accordance with the terms herein. Any Software or Derivative Works distributed shall be pursuant to a license agreement that contains all of the terms herein; and shall contain prominent notices stating how the Software, Derivative Works, or documentation was changed, the author and date of any such change and require acknowledgement of the orginal software/publicaitons by any users of the Derivative Works.
+
+d) You acknowledge that the Software is a valuable, proprietary asset of The Owners. You shall not market or sell the Software or Derivative Works.
+
+3. LICENSE EXCLUSIONS
+
+a) EXCEPT AS EXPRESSLY PROVIDED HEREIN, YOU SHALL MAKE NO OTHER USE OF THE SOFTWARE.
+
+b) You must obtain permission from The Licensor before receiving payment for distribution of or services using the Software or Derivative Works.
+
+c) You shall not allege or enjoin infringement or misappropriation by The Licensor in any Derivative Works, or by any third party obtaining Derivative Works, prepared by The Licensor and under license from The Licensor.
+4. TITLE AND PROTECTION OF SOFTWARE
+
+a) The Owners retains all title, right and interest to the Software and the underlying patents.
+
+b) Except for the Software, You retain all title, right and interest to the Derivative Works, subject to the terms of this Agreement.
+
+5. NO REPRESENTATIONS
+
+THE OWNERS DISCLAIMS ALL OTHER REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
+6. ATTRIBUTION
+
+a) You agree to retain and reproduce in all copies of Software the copyright and other proprietary notices and disclaimers as they appear in the Software, and keep intact all notices in the Software that refer to this License.
+
+b) You agree to provide attribution to the authors of this Software in any article based on research performed using Software or Derivative Works or with any distribution of Software or Derivative works.
+
+7. DEFAULT
+
+If YOU fail to perform any of its obligations under this Agreement, The Licensor, in addition to any other rights available to it under law or equity, may terminate this Agreement and the licenses granted hereunder by written notice to You. Unless otherwise provided in this Agreement, remedies shall be cumulative and there shall be no obligation to exercise a particular remedy.
+
+8. TERMINATION
+
+a) In the event that this Agreement is terminated, any sublicenses granted or Derivative Works distributed by Licensee shall remain in full force and effect.
+
+b) Within thirty (30) days of termination, You shall return to The Licensor or certify in writing to The Licensor that all copies or partial copies of Software in Your possession or control have been destroyed. c) In addition to this section, the sections entitled “Title and Protection of Software “No Representations” and “Limitation of Liability” shall survive termination of this Agreement.
+
+9. GENERAL
+a) No agency, partnership or employment is created by this Agreement.
+
+b) You may not use any of The Owners’ names, the terms in Meta-Recognition, or W-score in any advertising, public relations or media release without the prior written consent of the Owner.
+
+c) This Agreement shall be governed by the laws of the State of Colorado. Venue for any action or proceeding shall be Denver, Colorado. This Agreement constitutes the entire agreement between the parties and may only be modified by a written instrument signed by each parties authorized officers.
+
+If you accept this license please opt-in for and you will receive email with instructions. The email will also be used for update emails on future changes to the code.
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/libMR/Makefile b/Downstream/Open-Set-Action-Recognition/experiments/libMR/Makefile
new file mode 100644
index 0000000..8de7513
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/libMR/Makefile
@@ -0,0 +1,8 @@
+CXX= g++
+SRC= MetaRecognition.cpp weibull.c
+
+libmr: $(SRC) weibull.h malloc.h MetaRecognition.h
+ $(CXX) -o libmr $(SRC) -I.
+
+clean:
+ rm -f *~ *.o libmr
\ No newline at end of file
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/libMR/MetaRecognition.cpp b/Downstream/Open-Set-Action-Recognition/experiments/libMR/MetaRecognition.cpp
new file mode 100644
index 0000000..b423be0
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/libMR/MetaRecognition.cpp
@@ -0,0 +1,463 @@
+/*
+ * MetaRecognition.cpp
+ * Copyright 2011, Securics Inc.
+ See accompanying LICENSE agrement for details on rights.
+
+Parts of this technology are subject to SBIR data rights and as described in DFARS 252.227-7018 (June 1995) SBIR Data Rights which apply to Contract Number: N00014-11-C-0243 and STTR N00014-07-M-0421 to Securics Inc, 1870 Austin Bluffs Parkway, Colorado Springs, CO 80918
+
+The Government's rights to use, modify, reproduce, release, perform, display, or disclose technical data or computer software marked with this legend are restricted during the period shown as provided in paragraph (b)(4) of the Rights in Noncommercial Technical Data and Computer Software-Small Business Innovative Research (SBIR) Program clause contained in the above identified contract. Expiration of SBIR Data Rights: Expires four years after completion of the above cited project work for this or any other follow-on SBIR contract, whichever is later.
+
+No restrictions on government use apply after the expiration date shown above. Any reproduction of technical data, computer software, or portions thereof marked with this legend must also reproduce the markings.
+ *
+*/
+
+/** \mainpage
+
+
+ This library provides support for meta-recognition, i.e. recognizing when a recognition system is working well and when it is not and using that self-knowledge to improve the system. It can be used for prediction of failure, fusion, score renormalization, SVM renormalization and converting SVM or recognition scores into statistially well supported probility estimtes. The analysis is based on an analysis of the recognition system scores.
+
+
+The fundamental ideas are described in
+
+"Meta-Recognition: The Theory and Practice of Recognition Score Analysis,"
+Walter J. Scheirer, Anderson Rocha, Ross Micheals, Terrance E. Boult,
+IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI),
+33(8), pp 1689--1695, Aug, 2011.
+
+and SVM support as described in
+
+"Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search,"
+Walter J. Scheirer, Neeraj Kumar, Peter N. Belhumeur, Terrance E. Boult,
+Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
+June 2012.
+
+
+The underlying extream value theory provide stong theortical basis for the computations, but to make it useful one must transform the data into the proper frame. The C++ version provides objects that can compute and store information about the transform and then provide for prediction, w-score values (probability estimates), or renormalizatoin of a vector of data.
+
+ The library also contains a "C" interface functions for very basic weilbull usage for Meta-Recognition.
+ The C-based library has a number of STRONG assumptions you must follow as we cannot test for all of them.
+ 1) All fitting and testing are presuming "larger is better", If you are fitting something where smaller is better you need to transform it before fitting.
+ 2) All data is positive (okay we can and do test for that, but better to know upfront what you are doing)
+ 3) There must be sufficient range in your data to actually fit the weilbull. If all the data is the same, or nearly the same, it may fal to converge and will report errors.
+
+ While free for non-commercial use this library is subject to the license restrictions, see LICENSE.TXT for details.
+
+*/
+
+#include "MetaRecognition.h"
+#include
+//#include
+#include
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+extern int weibull_fit_verbose_debug;
+#ifdef __cplusplus
+}
+#endif
+MetaRecognition::MetaRecognition(int scores_to_dropx, int fitting_sizex, bool verb, double alphax, int translate_amountx):
+ scores_to_drop(scores_to_dropx),verbose(verb),fitting_size(fitting_sizex),alpha(alphax),translate_amount(translate_amountx)
+{
+ memset(parmhat,0,sizeof(parmhat));
+ memset(parmci,0,sizeof(parmci));
+ sign = 1;
+ ftype = complement_reject;
+ small_score=0;
+ isvalid=false;
+ if(verb) weibull_fit_verbose_debug=1;
+ else weibull_fit_verbose_debug=0;
+}
+
+MetaRecognition::~MetaRecognition()
+{
+ // free(parmhat);
+ // free(parmci);
+}
+
+bool MetaRecognition::is_valid(){
+ return isvalid;
+}
+
+void MetaRecognition::set_translate(double t){
+ translate_amount = t;
+ isvalid=false;
+};
+
+
+void MetaRecognition::Reset(){
+ memset(parmhat,0,sizeof(parmhat));
+ memset(parmci,0,sizeof(parmci));
+ sign = 1;
+ scores_to_drop = 0;
+ small_score=0;
+ isvalid=false;
+}
+
+
+int compare_sort_decending (const void * a, const void * b)
+{
+ const double *da = (const double *) a;
+ const double *db = (const double *) b;
+ return (*da < *db) - (*da > *db);
+}
+
+int compare_sort_assending (const void * a, const void * b)
+{
+ const double *da = (const double *) a;
+ const double *db = (const double *) b;
+ return (*da > *db) - (*da < *db);
+}
+
+inline const char * const BoolToString(bool b)
+{
+ return b ? "true" : "false";
+}
+
+inline int const BoolToInt(bool b)
+{
+ return b ? 1 : 0;
+}
+
+inline const bool IntToBool(const char * s)
+{
+ int val= atoi(s);
+ if(val) return true;
+ else return false;
+}
+
+//Wraps calls to real weibull_inv and weibull_cdf functions and handles properly translating the data passed
+//May eventually be a good idea to move real implementations of the functions here
+//IF we do away with the C implementation. For now this allows for backward compantiblity with
+//older code
+// Inv computes the scores of the inverse CDF, i.e. returns y such that CDF(y) =
+double MetaRecognition::Inv(double x)
+{
+ if(!isvalid) return -9999.0;
+ double score = weibull_inv(x, parmhat[0], parmhat[1]);
+ return (score - translate_amount + small_score)*sign;
+}
+
+double MetaRecognition::CDF(double x)
+{
+ if(!isvalid) return -9999.0;
+ double translated_x = x*sign + translate_amount - small_score;
+ double wscore=weibull_cdf(translated_x, parmhat[0], parmhat[1]);
+ if(ftype==complement_model || ftype==positive_model) return 1-wscore;
+ return wscore;
+};
+
+double MetaRecognition::W_score(double x){
+ return CDF(x);
+};
+
+bool MetaRecognition::Predict_Match(double x, double threshold){
+ double score = Inv(threshold);
+ if(sign <0) return (x < score);
+ return (x > score);
+};
+
+int MetaRecognition::ReNormalize(double *invec, double *outvec, int length)
+{
+ if(!isvalid) return -9997.0;
+ int rval=1;
+ for(int i=0; i< length; i++){
+ outvec[i] = W_score(invec[i]);
+ }
+ return rval;
+}
+
+
+//used by weibull__evt_low and weibull__evt_high, which sets the desired sign(low -1, high 1)
+//before passing to generic
+int MetaRecognition::EvtGeneric(double* inputData, int inputDataSize, int inward, double x)
+{
+ double * inputDataCopy = (double *) malloc(sizeof(double) * inputDataSize);
+
+ double * dataPtr = NULL;
+ int icnt=0;
+ if(!inward && (sign > 0) ) {
+ icnt = inputDataSize;
+ memcpy(inputDataCopy,inputData, inputDataSize*sizeof(double));
+ }
+ if(!inward && (sign < 0) ){
+ for(int i=0; i < inputDataSize; i++) inputDataCopy[i] = (inputData[i]*sign); //doing extremes just flip sign if needed
+ icnt = inputDataSize;
+ }
+ else if(inward && (sign < 0)) { /* this is fit above x but approaching x */
+ for(int i=0; i < inputDataSize; i++) {
+ if(inputData[i] > x) {
+ inputDataCopy[icnt++] = (inputData[i]*sign); //copy what is above x, and flip signs (so biggest is important)
+ }
+ }
+ } else if(inward && (sign > 0)) { /* this is fit below x but approaching x */
+ for(int i=0; i < inputDataSize; i++) {
+ if(inputData[i] < x) {
+ inputDataCopy[icnt++] = (inputData[i]); //copy only what is above x.
+ }
+ }
+ }
+
+ //sort data and get smallest score
+ qsort(inputDataCopy, icnt , sizeof(double), compare_sort_decending);
+
+ //Want only the top fitting_size scores but als noneed to adap if dropping top score
+ if(scores_to_drop>0){
+ dataPtr=inputDataCopy+scores_to_drop;
+ } else {
+ dataPtr=inputDataCopy;
+ }
+
+ small_score = dataPtr[fitting_size-1];
+
+ for(int i=0; i < fitting_size; i++)
+ {
+ //translate and subtract small score
+ dataPtr[i] = dataPtr[i] + translate_amount - small_score;
+ }
+
+
+ int rval = weibull_fit(parmhat, parmci, dataPtr, alpha, fitting_size);
+ isvalid= true;
+ if(rval != 1) Reset();
+ free(inputDataCopy);
+ return rval;
+}
+
+//Wrapper fitting functions EvtLow and EvtHigh to make it simpler for new users of the library.
+int MetaRecognition::FitLow(double* inputData, int inputDataSize, int fsize)
+{
+ if(fsize>0) fitting_size=fsize;
+ sign = -1;
+ return EvtGeneric(inputData, inputDataSize);
+}
+
+int MetaRecognition::FitHigh(double* inputData, int inputDataSize, int fsize)
+{
+ if(fsize>0) fitting_size=fsize;
+ sign = 1;
+ return EvtGeneric(inputData, inputDataSize);
+}
+
+int MetaRecognition::FitSVM(svm_node_libsvm* SVMdata, int inputDataSize, int label_of_interest, bool label_has_positive_score, int fit_type, int fit_size )
+{
+
+ Reset();
+ ftype = (MR_fitting_type)fit_type;
+ fitting_size = fit_size;
+ double * inputDataCopy = (double *) malloc(sizeof(double) * inputDataSize);
+ int sign_of_label_of_interest=0;
+ double * dataPtr = NULL;
+ int sign_of_expected_score=-1;
+ if(label_has_positive_score) sign_of_expected_score=1;
+
+ int icnt=0;
+ bool rejection=(ftype==complement_reject || ftype == positive_reject);
+ if(rejection) { // default we fit on the complement class and then do rejection to determine probability
+ for(int i=0; i < inputDataSize; i++) {
+ if(SVMdata[i].index != label_of_interest) inputDataCopy[icnt++] = (SVMdata[i].value); //doing extremes just flip sign if needed
+ else {
+ if(SVMdata[i].value >0) sign_of_label_of_interest++;
+ else sign_of_label_of_interest--;
+ }
+ }
+ } else {
+ for(int i=0; i < inputDataSize; i++) {
+ if(SVMdata[i].index == label_of_interest) inputDataCopy[icnt++] = (SVMdata[i].value); //doing extremes just flip sign if needed
+ else {
+ if(SVMdata[i].value >0) sign_of_label_of_interest++;
+ else sign_of_label_of_interest--;
+ }
+ }
+ }
+ if (verbose && sign_of_label_of_interest * sign_of_expected_score > 0){
+ printf("In MetaRecognition::FitSVM, warning: possible inconsistency average of the non-matching data has score %d, but expected sign is %d\n",
+ sign_of_label_of_interest, -sign_of_expected_score);
+ }
+
+
+ /* expected sign combines with reject_complement to determine if we have to flip or not.
+ We flip if positives scores, with smaller is better, is the goal,
+ we flip if sign_of_expected_score >0 and !force_rejection
+ we flip if sign_of_expected_score <0 and force_rejection */
+
+ if((!label_has_positive_score && rejection)
+ || (label_has_positive_score && !rejection)) {
+ sign = -1;
+ for(int i=0; i < icnt; i++) {
+ inputDataCopy[i] *= -1; //doing extremes just flip sign if needed
+ }
+ } else sign=1;
+
+ //sort data and get smallest score
+ qsort(inputDataCopy, icnt , sizeof(double), compare_sort_decending);
+
+ //Want only the top fitting_size scores but als noneed to adap if dropping top score
+ if(scores_to_drop){
+ dataPtr=inputDataCopy+scores_to_drop;
+ } else {
+ dataPtr=inputDataCopy;
+ }
+
+ small_score = dataPtr[fitting_size - 1];
+
+ for(int i=0; i < fitting_size; i++)
+ {
+ //translate and subtract small score
+ dataPtr[i] = dataPtr[i] + translate_amount - small_score;
+ }
+
+ int rval = weibull_fit(parmhat, parmci, dataPtr, alpha, fitting_size);
+
+ isvalid= true;
+ if(rval != 1) Reset();
+ free(inputDataCopy);
+ printf("Completed weibull fitting\n");
+ return rval;
+};
+
+void MetaRecognition::Save(std::ostream &outputStream) const
+{
+ if(outputStream.good() && isvalid)
+ {
+ try {
+ outputStream.precision(21);
+ outputStream.setf(std::ios::scientific,std::ios::floatfield);
+ outputStream << parmhat[0] << " " << parmhat[1] << " "
+ << parmci[0] << " " << parmci[1] << " "
+ << parmci[2] << " " << parmci[3] << " "
+ << sign << " "
+ << alpha << " "
+ << (int) ftype << " "
+ << fitting_size << " "
+ << translate_amount << " "
+ << small_score<< " "
+ << scores_to_drop
+ << std::endl;
+ } catch(std::bad_alloc& e) {
+ std::cout << "Could not allocate the required memory, failed with error: '" << e.what() << "'" << std::endl;
+ }
+ }
+}
+
+std::ostream& operator<< ( std::ostream& os, const MetaRecognition& mr )
+ {
+ mr.Save(os);
+ return os;
+ }
+
+std::istream& operator>> ( std::istream& is, MetaRecognition& mr )
+ {
+ mr.Load(is);
+ return is;
+ }
+
+
+void MetaRecognition::Load(std::istream &inputStream)
+{
+ isvalid=false;
+ int temp;
+ if(inputStream.good())
+ {
+ int iftype;
+ inputStream >> parmhat[0] >> parmhat[1]
+ >> parmci[0] >> parmci[1]
+ >> parmci[2] >> parmci[3]
+ >> sign
+ >> alpha
+ >> iftype
+ >> fitting_size
+ >> translate_amount
+ >> small_score
+ >> scores_to_drop;
+ isvalid=true;
+ ftype = (MR_fitting_type) iftype;
+ }
+}
+
+void MetaRecognition::Save(FILE *outputFile) const
+{
+ if((outputFile != NULL) && !feof(outputFile))
+ {
+ fprintf(outputFile,
+ "%21.18g %21.18g " //parmaht
+ "%21.18g %21.18g " //parmci
+ "%21.18g %21.18g "
+ "%d %f %d %d " //sign, alpha, fitting size
+ "%d %21.18g %d\n", //translate, small_score, scores_to_drop
+ parmhat[0], parmhat[1],
+ parmci[0],parmci[1],
+ parmci[2],parmci[3],
+ sign, alpha, (int) ftype,fitting_size,
+ translate_amount, small_score, scores_to_drop);
+ }
+}
+
+void MetaRecognition::Load(FILE *inputFile)
+{
+ int temp, iftype;
+ int retcode=0;
+ isvalid=false;
+ if((inputFile != NULL) && !feof(inputFile))
+ {
+
+ retcode = fscanf(inputFile,
+ "%lf %lf " //parmaht
+ "%lf %lf " //parmci
+ "%lf %lf "
+ "%d %lf %d %d " //sign, alpha, fitting size
+ "%d %lf %d ", //translate, small_score, scores_to_drop,
+ parmhat, parmhat+1,
+ parmci,parmci+1,
+ parmci+2,parmci+3,
+ &sign, &alpha, &iftype, &fitting_size,
+ &translate_amount, &small_score, &scores_to_drop);
+ isvalid=true;
+ ftype = (MR_fitting_type) iftype;
+ }
+}
+
+
+void MetaRecognition::Save(char* filename) const
+{
+ FILE* fp = fopen(filename,"w");
+ if(fp) {
+ Save(fp);
+ fclose(fp);
+ } else if(strlen(filename)>0)
+ fprintf(stderr,"SaveWeibull could not open file |%s|\n",filename);
+ else fprintf(stderr,"SaveWeibull called with null filename\n");
+}
+
+void MetaRecognition::Load(char* filename){
+ FILE* fp = fopen(filename,"r");
+ isvalid=false;
+ if(fp) {
+ Load(fp);
+ isvalid=true;
+ fclose(fp);
+ } else if(strlen(filename)>0)
+ fprintf(stderr,"LoadWeibull could not open file |%s|\n",filename);
+ else fprintf(stderr,"LoadWeibull called with null filename\n");
+
+}
+
+std::string MetaRecognition::to_string() {
+ std::stringstream oss;
+ this->Save(oss);
+ return oss.str();
+}
+void MetaRecognition::from_string(std::string input) {
+ std::stringstream iss(input);
+ this->Load(iss);
+}
+
+
+int MetaRecognition::set_fitting_size(int nsize){ isvalid=false; return fitting_size=nsize;}
+int MetaRecognition::get_fitting_size(){ return fitting_size;}
+int MetaRecognition::get_translate_amount(){ return translate_amount;}
+int MetaRecognition::set_translate_amount(int ntrans) {isvalid=false; return translate_amount=ntrans;}
+double MetaRecognition::get_small_score(){return small_score;}
+double MetaRecognition::set_small_score(double nscore){isvalid=false; return small_score=nscore;}
+int MetaRecognition::get_sign(){return sign;}
+int MetaRecognition::set_sign(int nsign){return sign=nsign;}
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/libMR/MetaRecognition.h b/Downstream/Open-Set-Action-Recognition/experiments/libMR/MetaRecognition.h
new file mode 100644
index 0000000..76088ec
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/libMR/MetaRecognition.h
@@ -0,0 +1,148 @@
+/**
+ * MetaRecognition.h:
+
+ * @Author Terry Boult tboult at securics com
+ * @Author Vijay Iyer viyer at securics com
+
+ *
+ * Copyright 2010, 2011, Securics Inc.
+
+ * Copyright 2011, Securics Inc.
+ See accompanying LICENSE agrement for details on rights.
+
+Parts of this technology are subject to SBIR data rights and as described in DFARS 252.227-7018 (June 1995) SBIR Data Rights which apply to Contract Number: N00014-11-C-0243 and STTR N00014-07-M-0421 to Securics Inc, 1870 Austin Bluffs Parkway, Colorado Springs, CO 80918
+
+The Government's rights to use, modify, reproduce, release, perform, display, or disclose technical data or computer software marked with this legend are restricted during the period shown as provided in paragraph (b)(4) of the Rights in Noncommercial Technical Data and Computer Software-Small Business Innovative Research (SBIR) Program clause contained in the above identified contract. Expiration of SBIR Data Rights: Expires four years after completion of the above cited project work for this or any other follow-on SBIR contract, whichever is later.
+
+No restrictions on government use apply after the expiration date shown above. Any reproduction of technical data, computer software, or portions thereof marked with this legend must also reproduce the markings.
+ *
+*/
+
+#pragma once
+#ifndef MetaRecognition_H
+#define MetaRecognition_H
+
+
+#ifdef HAVE_CONFIG_H
+# include "config.h"
+#endif
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+
+#include "weibull.h"
+
+#ifdef _WIN32
+#define DLLEXPORT _declspec(dllexport)
+#else
+#define DLLEXPORT
+#endif
+
+#define MAX_LINE 256
+
+/// structure for svm data used by libSVM, used to allow easy MetaRecognition for SVM results (used as an argument for MetaRecogniton::fitSVM)
+struct svm_node_libsvm
+{
+ int index; //!< class label, classic is -1 for negative class add +1 for positive class, but can be general its for multi-class
+ double value;//!< the SVM decision score
+};
+
+/**!
+ Class MetaRecognition provides a object-based interface for Meta-Recognition. The object can be ...
+
+ TBD
+
+
+*/
+class DLLEXPORT MetaRecognition //! Primary object/methods for tranforming and computing needed for any Meta recogntion task
+{
+public:
+
+/** Ctor, can call with no arguments (uses default arguments for construciton).
+ All space is on the stack.
+ Object will exist but is not valid until some fitting fucntion is called
+*/
+
+ MetaRecognition( int scores_to_drop=0, //!< is this object for prediction, if so how many top scores to drop when fitting
+ int fitting_size=9, //!< tail size for fitting. With small data the defaults are fine.. if you have millions make it larger for better predictions
+ bool verbose = false, //!< is the code chatty on errors during fitting, useful for debugging
+ double alpha=5.0, //!< band for confidence interfals
+ int translate_amount=10000 //!< shifting data to ensure all is positive.. if data is very broad and you want some probabilities for all points you can make it larger..
+ );
+
+ ~MetaRecognition();
+
+ bool is_valid(); //!< is this object valid..i.e. has data been properly fit to determine parameters.
+ void set_translate(double t); //!< Change translate_amount to x, invalidates object
+
+ void Reset(); //!< reset to "invalid" state
+
+ bool Predict_Match(double x, double threshold = .9999999); //!< Is X from the "match" distribution (i.e. we reject null hypothesis of non-match),
+ double W_score(double x); //!< This is the commonly used function.. after fitting, it returns the probability of the given score being "correct". It is the same as CDF
+ double CDF(double x); //!< This is the cummumlative probablity of match being corrrect (or more precisely the probility the score (after transform) being an outlier for the distribution, which given the transforms applied, so bigger is better, this is the probablity the score is correct.
+ double Inv(double p); //!< This is score for which one would obtain CDF probability p (i.e. x such that p = CDF(x))
+
+ int ReNormalize(double *invec, double *outvec, int length); //!< W-score Renormalize the vecotor invec[0:length-1] into outvec (in and out can be same) return is 1 for success, <0 for error code
+
+
+ /// Use FitHight if your data is such that is larger is better. The code will still transform, and keep parmeters to keep small data away from zero.
+ // If you get scores that are complain about it being negative, make a MR object with different (larger) translate amount
+ /// returns 1 for success, <0 for error code
+ int FitHigh(double* inputData, int inputDataSize, int fit_size=-1);
+
+ ///Use FitLow if your data is such that smaller scores are better.. we'll transform it for you and keep the
+ ///transform parameters in the class so later calls to W_score or CDF do the right thing.
+ /// returns 1 for success, <0 for error code
+ int FitLow(double* inputData, int inputDataSize, int fit_size=-1);//
+
+ /// the types of fitting supported for SVM modeling
+ typedef enum {complement_reject=1, positive_reject=2, complement_model=3, positive_model=4} MR_fitting_type;
+
+ /// The function to use if you have SVM data, it separated out the data for the label of interst (or rejecting
+ /// the complement of that label, which is the default) and uses that for fitting.
+ /// Returns 1 if it worked, <0 for error codes.
+ int FitSVM(svm_node_libsvm* SVMdata, int inputDataSize, int label_of_interest =1, bool label_has_positive_score=true, int fit_type = 1, int fit_size=9 );
+
+
+ friend std::ostream& operator<<( std::ostream&, const MetaRecognition& ); //!< various I/O functions
+ friend std::istream& operator>>( std::istream&, MetaRecognition& ); //!< various I/O functions
+
+ void Save(std::ostream &outputStream) const; //!< various I/O functions
+ void Load(std::istream &inputStream); //!< various I/O functions
+ void Save(FILE *outputFile) const; //!< various I/O functions
+ void Load(FILE *inputFile); //!< various I/O functions
+ void Save(char* filename) const; //!< various I/O functions
+ void Load(char* filename); //!< various I/O functions
+ int get_fitting_size(); //!< Get get_fitting_size (aka tail size)
+ int set_fitting_size(int nsize); //!< reset object and define new fitting size
+ int get_translate_amount(); //!< Get get_internal tranlation amount (you probably don't need this, but just in case)
+ int set_translate_amount(int ntrans); //!< reset object and define new translate amount.. if you get errors because of negative data, increase this
+ int get_sign(); //!< Get get_internal sign variable. (you probably don't need this, but just in case)
+ int set_sign(int nsign); //!< reset object and set sign (you probably don't need this, but just in case)
+ double get_small_score(); //!< Get get_internal smaller translation amount (you probably don't need this, but just in case)
+ double set_small_score(double nscore); //!< reset object and reset internal smaller translation amount (you probably don't need this, but just in case)
+ bool verbose; //!< do we print internal/debugging stuff. Default is false. (you probably don't need this, but just in case)
+ std::string to_string(); //!< Convert this object to a C++ string
+ void from_string(std::string in); //!< Convert this object from a C++ string
+
+protected:
+ int EvtGeneric(double* inputData, int inputDataSize, int fit_inward=0, double x=0);
+ double parmhat[2]; //!< parameters of the Weibull, scale then shape
+ double parmci[4]; //!< confidence interval for parms scale high, scale low, shape high, shape low
+ double alpha; //!< parameter for estimation of size of confidence interval
+ int sign; //!< sign is postive is larger is better, negative means orginally smaller was better (we transformed for fitting).
+ MR_fitting_type ftype; //!< type of fitting used for SVM.. default is reject complement
+ int fitting_size; //!< tail size for fitting in any of the FitXX functions
+ int translate_amount; //!< we transform data so all fittng data data is positive and bigger is better, this predefined constant helps ensure more of the end-user data is non-negative.
+ double small_score; //!< the smallest score, so all fitting data is consistently postive. part of our transform
+ int scores_to_drop; //!< when fitting for recognition prediction, how many top score are hypothesized to be a match, so we can fit on non-match data. Only used in for fitting, no impact on transform.
+ bool isvalid; //!< is the parameters in the object valid. private:
+
+};
+
+#endif
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/libMR/build_libmr_python.sh b/Downstream/Open-Set-Action-Recognition/experiments/libMR/build_libmr_python.sh
new file mode 100644
index 0000000..7111afb
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/libMR/build_libmr_python.sh
@@ -0,0 +1,28 @@
+#!/bin/bash
+
+# This script creates a clean temporary environment Python, and then
+# builds LibMR's python bindings.
+
+if [ '!' -f setup.py ]; then
+ echo Put this script into the same folder as setup.py
+ exit 1
+fi
+
+echo Step 1: Download virtualenv
+wget -O virtualenv-1.9.1.tar.gz --no-check-certificate https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.1.tar.gz
+tar xvf virtualenv-1.9.1.tar.gz
+
+echo Step 2: Create virtualenv
+python virtualenv-1.9.1/virtualenv.py --system-site-packages venv
+
+echo Step 3: Entering virtualenv and installing dependencies
+source venv/bin/activate
+pip install cython==0.19.1
+
+echo Step 5: Build the extension
+rm -f python/libmr.cpp
+python setup.py build_ext -i
+
+deactivate
+
+echo The .so should be built in the current folder.
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/libMR/compile.sh b/Downstream/Open-Set-Action-Recognition/experiments/libMR/compile.sh
new file mode 100644
index 0000000..d3d6454
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/libMR/compile.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+echo "----- Removing previously compiled libmr.so -----\n"
+rm -r build
+rm *.model
+rm libmr.so
+rm *.dump
+rm ../libmr.so
+
+echo "----- Building and compiling libmr ------- \n"
+python setup.py build_ext -i
+# cp libmr.so ../
+
+# echo "----- Completed Compiling libmr -------- \n"
+# echo "Now trying python -c \"import libmr\""
+# python test_libmr.py
+# echo "----- Compiling Done. Now import *.so file in your application -----\n"
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/libMR/estimate_wscores.py b/Downstream/Open-Set-Action-Recognition/experiments/libMR/estimate_wscores.py
new file mode 100644
index 0000000..944786b
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/libMR/estimate_wscores.py
@@ -0,0 +1,26 @@
+import os, sys
+import scipy as sp
+import libmr
+
+def main():
+
+ posscores = sp.asarray([0.245 , 0.2632, 0.3233, 0.3573, 0.4014, 0.4055, 0.4212, 0.5677])
+ test_distances = sp.asarray([ 0.05, 0.1 , 0.25, 0.4 , 0.75, 1. , 1.5 , 2.])
+
+ mr = libmr.MR()
+ # since higher is worse and we want to fit the higher tail,
+ # use fit_high()
+ mr.fit_high(posscores, posscores.shape[0])
+ wscores = mr.w_score_vector(test_distances)
+ for i in range(wscores.shape[0]):
+ print "%.2f %.2f %.2f" %(test_distances[i], wscores[i], mr.inv(wscores[i]))
+ # wscores are the ones to be used in the equation
+ # s_i * (1 - rho_i)
+ print "Low wscore --> Low probability that the score is outlier i.e. sample IS NOT outlier"
+ print "High wscore --> High probability that the score is outlier i.e. sample IS an outlier"
+ print "posscores: ", posscores
+ print "test_distances: ", test_distances
+ print "wscores: ", wscores
+
+if __name__ == "__main__":
+ main()
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/libMR/libmr.c b/Downstream/Open-Set-Action-Recognition/experiments/libMR/libmr.c
new file mode 100644
index 0000000..06f2230
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/libMR/libmr.c
@@ -0,0 +1 @@
+#error Do not use this file, it is the result of a failed Cython compilation.
diff --git a/Downstream/Open-Set-Action-Recognition/experiments/libMR/libmr.cpp b/Downstream/Open-Set-Action-Recognition/experiments/libMR/libmr.cpp
new file mode 100644
index 0000000..bc6a2d8
--- /dev/null
+++ b/Downstream/Open-Set-Action-Recognition/experiments/libMR/libmr.cpp
@@ -0,0 +1,26539 @@
+/* Generated by Cython 0.29.22 */
+
+/* BEGIN: Cython Metadata
+{
+ "distutils": {
+ "depends": [
+ "/home/wentao/anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h",
+ "/home/wentao/anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/core/include/numpy/ufuncobject.h",
+ "MetaRecognition.h"
+ ],
+ "include_dirs": [
+ ".",
+ "/home/wentao/anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/core/include"
+ ],
+ "language": "c++",
+ "name": "libmr",
+ "sources": [
+ "libmr.pyx",
+ "MetaRecognition.cpp",
+ "weibull.c"
+ ]
+ },
+ "module_name": "libmr"
+}
+END: Cython Metadata */
+
+#define PY_SSIZE_T_CLEAN
+#include "Python.h"
+#ifndef Py_PYTHON_H
+ #error Python headers needed to compile C extensions, please install development version of Python.
+#elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000)
+ #error Cython requires Python 2.6+ or Python 3.3+.
+#else
+#define CYTHON_ABI "0_29_22"
+#define CYTHON_HEX_VERSION 0x001D16F0
+#define CYTHON_FUTURE_DIVISION 0
+#include
+#ifndef offsetof
+ #define offsetof(type, member) ( (size_t) & ((type*)0) -> member )
+#endif
+#if !defined(WIN32) && !defined(MS_WINDOWS)
+ #ifndef __stdcall
+ #define __stdcall
+ #endif
+ #ifndef __cdecl
+ #define __cdecl
+ #endif
+ #ifndef __fastcall
+ #define __fastcall
+ #endif
+#endif
+#ifndef DL_IMPORT
+ #define DL_IMPORT(t) t
+#endif
+#ifndef DL_EXPORT
+ #define DL_EXPORT(t) t
+#endif
+#define __PYX_COMMA ,
+#ifndef HAVE_LONG_LONG
+ #if PY_VERSION_HEX >= 0x02070000
+ #define HAVE_LONG_LONG
+ #endif
+#endif
+#ifndef PY_LONG_LONG
+ #define PY_LONG_LONG LONG_LONG
+#endif
+#ifndef Py_HUGE_VAL
+ #define Py_HUGE_VAL HUGE_VAL
+#endif
+#ifdef PYPY_VERSION
+ #define CYTHON_COMPILING_IN_PYPY 1
+ #define CYTHON_COMPILING_IN_PYSTON 0
+ #define CYTHON_COMPILING_IN_CPYTHON 0
+ #undef CYTHON_USE_TYPE_SLOTS
+ #define CYTHON_USE_TYPE_SLOTS 0
+ #undef CYTHON_USE_PYTYPE_LOOKUP
+ #define CYTHON_USE_PYTYPE_LOOKUP 0
+ #if PY_VERSION_HEX < 0x03050000
+ #undef CYTHON_USE_ASYNC_SLOTS
+ #define CYTHON_USE_ASYNC_SLOTS 0
+ #elif !defined(CYTHON_USE_ASYNC_SLOTS)
+ #define CYTHON_USE_ASYNC_SLOTS 1
+ #endif
+ #undef CYTHON_USE_PYLIST_INTERNALS
+ #define CYTHON_USE_PYLIST_INTERNALS 0
+ #undef CYTHON_USE_UNICODE_INTERNALS
+ #define CYTHON_USE_UNICODE_INTERNALS 0
+ #undef CYTHON_USE_UNICODE_WRITER
+ #define CYTHON_USE_UNICODE_WRITER 0
+ #undef CYTHON_USE_PYLONG_INTERNALS
+ #define CYTHON_USE_PYLONG_INTERNALS 0
+ #undef CYTHON_AVOID_BORROWED_REFS
+ #define CYTHON_AVOID_BORROWED_REFS 1
+ #undef CYTHON_ASSUME_SAFE_MACROS
+ #define CYTHON_ASSUME_SAFE_MACROS 0
+ #undef CYTHON_UNPACK_METHODS
+ #define CYTHON_UNPACK_METHODS 0
+ #undef CYTHON_FAST_THREAD_STATE
+ #define CYTHON_FAST_THREAD_STATE 0
+ #undef CYTHON_FAST_PYCALL
+ #define CYTHON_FAST_PYCALL 0
+ #undef CYTHON_PEP489_MULTI_PHASE_INIT
+ #define CYTHON_PEP489_MULTI_PHASE_INIT 0
+ #undef CYTHON_USE_TP_FINALIZE
+ #define CYTHON_USE_TP_FINALIZE 0
+ #undef CYTHON_USE_DICT_VERSIONS
+ #define CYTHON_USE_DICT_VERSIONS 0
+ #undef CYTHON_USE_EXC_INFO_STACK
+ #define CYTHON_USE_EXC_INFO_STACK 0
+#elif defined(PYSTON_VERSION)
+ #define CYTHON_COMPILING_IN_PYPY 0
+ #define CYTHON_COMPILING_IN_PYSTON 1
+ #define CYTHON_COMPILING_IN_CPYTHON 0
+ #ifndef CYTHON_USE_TYPE_SLOTS
+ #define CYTHON_USE_TYPE_SLOTS 1
+ #endif
+ #undef CYTHON_USE_PYTYPE_LOOKUP
+ #define CYTHON_USE_PYTYPE_LOOKUP 0
+ #undef CYTHON_USE_ASYNC_SLOTS
+ #define CYTHON_USE_ASYNC_SLOTS 0
+ #undef CYTHON_USE_PYLIST_INTERNALS
+ #define CYTHON_USE_PYLIST_INTERNALS 0
+ #ifndef CYTHON_USE_UNICODE_INTERNALS
+ #define CYTHON_USE_UNICODE_INTERNALS 1
+ #endif
+ #undef CYTHON_USE_UNICODE_WRITER
+ #define CYTHON_USE_UNICODE_WRITER 0
+ #undef CYTHON_USE_PYLONG_INTERNALS
+ #define CYTHON_USE_PYLONG_INTERNALS 0
+ #ifndef CYTHON_AVOID_BORROWED_REFS
+ #define CYTHON_AVOID_BORROWED_REFS 0
+ #endif
+ #ifndef CYTHON_ASSUME_SAFE_MACROS
+ #define CYTHON_ASSUME_SAFE_MACROS 1
+ #endif
+ #ifndef CYTHON_UNPACK_METHODS
+ #define CYTHON_UNPACK_METHODS 1
+ #endif
+ #undef CYTHON_FAST_THREAD_STATE
+ #define CYTHON_FAST_THREAD_STATE 0
+ #undef CYTHON_FAST_PYCALL
+ #define CYTHON_FAST_PYCALL 0
+ #undef CYTHON_PEP489_MULTI_PHASE_INIT
+ #define CYTHON_PEP489_MULTI_PHASE_INIT 0
+ #undef CYTHON_USE_TP_FINALIZE
+ #define CYTHON_USE_TP_FINALIZE 0
+ #undef CYTHON_USE_DICT_VERSIONS
+ #define CYTHON_USE_DICT_VERSIONS 0
+ #undef CYTHON_USE_EXC_INFO_STACK
+ #define CYTHON_USE_EXC_INFO_STACK 0
+#else
+ #define CYTHON_COMPILING_IN_PYPY 0
+ #define CYTHON_COMPILING_IN_PYSTON 0
+ #define CYTHON_COMPILING_IN_CPYTHON 1
+ #ifndef CYTHON_USE_TYPE_SLOTS
+ #define CYTHON_USE_TYPE_SLOTS 1
+ #endif
+ #if PY_VERSION_HEX < 0x02070000
+ #undef CYTHON_USE_PYTYPE_LOOKUP
+ #define CYTHON_USE_PYTYPE_LOOKUP 0
+ #elif !defined(CYTHON_USE_PYTYPE_LOOKUP)
+ #define CYTHON_USE_PYTYPE_LOOKUP 1
+ #endif
+ #if PY_MAJOR_VERSION < 3
+ #undef CYTHON_USE_ASYNC_SLOTS
+ #define CYTHON_USE_ASYNC_SLOTS 0
+ #elif !defined(CYTHON_USE_ASYNC_SLOTS)
+ #define CYTHON_USE_ASYNC_SLOTS 1
+ #endif
+ #if PY_VERSION_HEX < 0x02070000
+ #undef CYTHON_USE_PYLONG_INTERNALS
+ #define CYTHON_USE_PYLONG_INTERNALS 0
+ #elif !defined(CYTHON_USE_PYLONG_INTERNALS)
+ #define CYTHON_USE_PYLONG_INTERNALS 1
+ #endif
+ #ifndef CYTHON_USE_PYLIST_INTERNALS
+ #define CYTHON_USE_PYLIST_INTERNALS 1
+ #endif
+ #ifndef CYTHON_USE_UNICODE_INTERNALS
+ #define CYTHON_USE_UNICODE_INTERNALS 1
+ #endif
+ #if PY_VERSION_HEX < 0x030300F0
+ #undef CYTHON_USE_UNICODE_WRITER
+ #define CYTHON_USE_UNICODE_WRITER 0
+ #elif !defined(CYTHON_USE_UNICODE_WRITER)
+ #define CYTHON_USE_UNICODE_WRITER 1
+ #endif
+ #ifndef CYTHON_AVOID_BORROWED_REFS
+ #define CYTHON_AVOID_BORROWED_REFS 0
+ #endif
+ #ifndef CYTHON_ASSUME_SAFE_MACROS
+ #define CYTHON_ASSUME_SAFE_MACROS 1
+ #endif
+ #ifndef CYTHON_UNPACK_METHODS
+ #define CYTHON_UNPACK_METHODS 1
+ #endif
+ #ifndef CYTHON_FAST_THREAD_STATE
+ #define CYTHON_FAST_THREAD_STATE 1
+ #endif
+ #ifndef CYTHON_FAST_PYCALL
+ #define CYTHON_FAST_PYCALL 1
+ #endif
+ #ifndef CYTHON_PEP489_MULTI_PHASE_INIT
+ #define CYTHON_PEP489_MULTI_PHASE_INIT (PY_VERSION_HEX >= 0x03050000)
+ #endif
+ #ifndef CYTHON_USE_TP_FINALIZE
+ #define CYTHON_USE_TP_FINALIZE (PY_VERSION_HEX >= 0x030400a1)
+ #endif
+ #ifndef CYTHON_USE_DICT_VERSIONS
+ #define CYTHON_USE_DICT_VERSIONS (PY_VERSION_HEX >= 0x030600B1)
+ #endif
+ #ifndef CYTHON_USE_EXC_INFO_STACK
+ #define CYTHON_USE_EXC_INFO_STACK (PY_VERSION_HEX >= 0x030700A3)
+ #endif
+#endif
+#if !defined(CYTHON_FAST_PYCCALL)
+#define CYTHON_FAST_PYCCALL (CYTHON_FAST_PYCALL && PY_VERSION_HEX >= 0x030600B1)
+#endif
+#if CYTHON_USE_PYLONG_INTERNALS
+ #include "longintrepr.h"
+ #undef SHIFT
+ #undef BASE
+ #undef MASK
+ #ifdef SIZEOF_VOID_P
+ enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*)) };
+ #endif
+#endif
+#ifndef __has_attribute
+ #define __has_attribute(x) 0
+#endif
+#ifndef __has_cpp_attribute
+ #define __has_cpp_attribute(x) 0
+#endif
+#ifndef CYTHON_RESTRICT
+ #if defined(__GNUC__)
+ #define CYTHON_RESTRICT __restrict__
+ #elif defined(_MSC_VER) && _MSC_VER >= 1400
+ #define CYTHON_RESTRICT __restrict
+ #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L
+ #define CYTHON_RESTRICT restrict
+ #else
+ #define CYTHON_RESTRICT
+ #endif
+#endif
+#ifndef CYTHON_UNUSED
+# if defined(__GNUC__)
+# if !(defined(__cplusplus)) || (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 4))
+# define CYTHON_UNUSED __attribute__ ((__unused__))
+# else
+# define CYTHON_UNUSED
+# endif
+# elif defined(__ICC) || (defined(__INTEL_COMPILER) && !defined(_MSC_VER))
+# define CYTHON_UNUSED __attribute__ ((__unused__))
+# else
+# define CYTHON_UNUSED
+# endif
+#endif
+#ifndef CYTHON_MAYBE_UNUSED_VAR
+# if defined(__cplusplus)
+ template void CYTHON_MAYBE_UNUSED_VAR( const T& ) { }
+# else
+# define CYTHON_MAYBE_UNUSED_VAR(x) (void)(x)
+# endif
+#endif
+#ifndef CYTHON_NCP_UNUSED
+# if CYTHON_COMPILING_IN_CPYTHON
+# define CYTHON_NCP_UNUSED
+# else
+# define CYTHON_NCP_UNUSED CYTHON_UNUSED
+# endif
+#endif
+#define __Pyx_void_to_None(void_result) ((void)(void_result), Py_INCREF(Py_None), Py_None)
+#ifdef _MSC_VER
+ #ifndef _MSC_STDINT_H_
+ #if _MSC_VER < 1300
+ typedef unsigned char uint8_t;
+ typedef unsigned int uint32_t;
+ #else
+ typedef unsigned __int8 uint8_t;
+ typedef unsigned __int32 uint32_t;
+ #endif
+ #endif
+#else
+ #include
+#endif
+#ifndef CYTHON_FALLTHROUGH
+ #if defined(__cplusplus) && __cplusplus >= 201103L
+ #if __has_cpp_attribute(fallthrough)
+ #define CYTHON_FALLTHROUGH [[fallthrough]]
+ #elif __has_cpp_attribute(clang::fallthrough)
+ #define CYTHON_FALLTHROUGH [[clang::fallthrough]]
+ #elif __has_cpp_attribute(gnu::fallthrough)
+ #define CYTHON_FALLTHROUGH [[gnu::fallthrough]]
+ #endif
+ #endif
+ #ifndef CYTHON_FALLTHROUGH
+ #if __has_attribute(fallthrough)
+ #define CYTHON_FALLTHROUGH __attribute__((fallthrough))
+ #else
+ #define CYTHON_FALLTHROUGH
+ #endif
+ #endif
+ #if defined(__clang__ ) && defined(__apple_build_version__)
+ #if __apple_build_version__ < 7000000
+ #undef CYTHON_FALLTHROUGH
+ #define CYTHON_FALLTHROUGH
+ #endif
+ #endif
+#endif
+
+#ifndef __cplusplus
+ #error "Cython files generated with the C++ option must be compiled with a C++ compiler."
+#endif
+#ifndef CYTHON_INLINE
+ #if defined(__clang__)
+ #define CYTHON_INLINE __inline__ __attribute__ ((__unused__))
+ #else
+ #define CYTHON_INLINE inline
+ #endif
+#endif
+template
+void __Pyx_call_destructor(T& x) {
+ x.~T();
+}
+template
+class __Pyx_FakeReference {
+ public:
+ __Pyx_FakeReference() : ptr(NULL) { }
+ __Pyx_FakeReference(const T& ref) : ptr(const_cast(&ref)) { }
+ T *operator->() { return ptr; }
+ T *operator&() { return ptr; }
+ operator T&() { return *ptr; }
+ template bool operator ==(U other) { return *ptr == other; }
+ template bool operator !=(U other) { return *ptr != other; }
+ private:
+ T *ptr;
+};
+
+#if CYTHON_COMPILING_IN_PYPY && PY_VERSION_HEX < 0x02070600 && !defined(Py_OptimizeFlag)
+ #define Py_OptimizeFlag 0
+#endif
+#define __PYX_BUILD_PY_SSIZE_T "n"
+#define CYTHON_FORMAT_SSIZE_T "z"
+#if PY_MAJOR_VERSION < 3
+ #define __Pyx_BUILTIN_MODULE_NAME "__builtin__"
+ #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\
+ PyCode_New(a+k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)
+ #define __Pyx_DefaultClassType PyClass_Type
+#else
+ #define __Pyx_BUILTIN_MODULE_NAME "builtins"
+#if PY_VERSION_HEX >= 0x030800A4 && PY_VERSION_HEX < 0x030800B2
+ #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\
+ PyCode_New(a, 0, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)
+#else
+ #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\
+ PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)
+#endif
+ #define __Pyx_DefaultClassType PyType_Type
+#endif
+#ifndef Py_TPFLAGS_CHECKTYPES
+ #define Py_TPFLAGS_CHECKTYPES 0
+#endif
+#ifndef Py_TPFLAGS_HAVE_INDEX
+ #define Py_TPFLAGS_HAVE_INDEX 0
+#endif
+#ifndef Py_TPFLAGS_HAVE_NEWBUFFER
+ #define Py_TPFLAGS_HAVE_NEWBUFFER 0
+#endif
+#ifndef Py_TPFLAGS_HAVE_FINALIZE
+ #define Py_TPFLAGS_HAVE_FINALIZE 0
+#endif
+#ifndef METH_STACKLESS
+ #define METH_STACKLESS 0
+#endif
+#if PY_VERSION_HEX <= 0x030700A3 || !defined(METH_FASTCALL)
+ #ifndef METH_FASTCALL
+ #define METH_FASTCALL 0x80
+ #endif
+ typedef PyObject *(*__Pyx_PyCFunctionFast) (PyObject *self, PyObject *const *args, Py_ssize_t nargs);
+ typedef PyObject *(*__Pyx_PyCFunctionFastWithKeywords) (PyObject *self, PyObject *const *args,
+ Py_ssize_t nargs, PyObject *kwnames);
+#else
+ #define __Pyx_PyCFunctionFast _PyCFunctionFast
+ #define __Pyx_PyCFunctionFastWithKeywords _PyCFunctionFastWithKeywords
+#endif
+#if CYTHON_FAST_PYCCALL
+#define __Pyx_PyFastCFunction_Check(func)\
+ ((PyCFunction_Check(func) && (METH_FASTCALL == (PyCFunction_GET_FLAGS(func) & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS)))))
+#else
+#define __Pyx_PyFastCFunction_Check(func) 0
+#endif
+#if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Malloc)
+ #define PyObject_Malloc(s) PyMem_Malloc(s)
+ #define PyObject_Free(p) PyMem_Free(p)
+ #define PyObject_Realloc(p) PyMem_Realloc(p)
+#endif
+#if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX < 0x030400A1
+ #define PyMem_RawMalloc(n) PyMem_Malloc(n)
+ #define PyMem_RawRealloc(p, n) PyMem_Realloc(p, n)
+ #define PyMem_RawFree(p) PyMem_Free(p)
+#endif
+#if CYTHON_COMPILING_IN_PYSTON
+ #define __Pyx_PyCode_HasFreeVars(co) PyCode_HasFreeVars(co)
+ #define __Pyx_PyFrame_SetLineNumber(frame, lineno) PyFrame_SetLineNumber(frame, lineno)
+#else
+ #define __Pyx_PyCode_HasFreeVars(co) (PyCode_GetNumFree(co) > 0)
+ #define __Pyx_PyFrame_SetLineNumber(frame, lineno) (frame)->f_lineno = (lineno)
+#endif
+#if !CYTHON_FAST_THREAD_STATE || PY_VERSION_HEX < 0x02070000
+ #define __Pyx_PyThreadState_Current PyThreadState_GET()
+#elif PY_VERSION_HEX >= 0x03060000
+ #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet()
+#elif PY_VERSION_HEX >= 0x03000000
+ #define __Pyx_PyThreadState_Current PyThreadState_GET()
+#else
+ #define __Pyx_PyThreadState_Current _PyThreadState_Current
+#endif
+#if PY_VERSION_HEX < 0x030700A2 && !defined(PyThread_tss_create) && !defined(Py_tss_NEEDS_INIT)
+#include "pythread.h"
+#define Py_tss_NEEDS_INIT 0
+typedef int Py_tss_t;
+static CYTHON_INLINE int PyThread_tss_create(Py_tss_t *key) {
+ *key = PyThread_create_key();
+ return 0;
+}
+static CYTHON_INLINE Py_tss_t * PyThread_tss_alloc(void) {
+ Py_tss_t *key = (Py_tss_t *)PyObject_Malloc(sizeof(Py_tss_t));
+ *key = Py_tss_NEEDS_INIT;
+ return key;
+}
+static CYTHON_INLINE void PyThread_tss_free(Py_tss_t *key) {
+ PyObject_Free(key);
+}
+static CYTHON_INLINE int PyThread_tss_is_created(Py_tss_t *key) {
+ return *key != Py_tss_NEEDS_INIT;
+}
+static CYTHON_INLINE void PyThread_tss_delete(Py_tss_t *key) {
+ PyThread_delete_key(*key);
+ *key = Py_tss_NEEDS_INIT;
+}
+static CYTHON_INLINE int PyThread_tss_set(Py_tss_t *key, void *value) {
+ return PyThread_set_key_value(*key, value);
+}
+static CYTHON_INLINE void * PyThread_tss_get(Py_tss_t *key) {
+ return PyThread_get_key_value(*key);
+}
+#endif
+#if CYTHON_COMPILING_IN_CPYTHON || defined(_PyDict_NewPresized)
+#define __Pyx_PyDict_NewPresized(n) ((n <= 8) ? PyDict_New() : _PyDict_NewPresized(n))
+#else
+#define __Pyx_PyDict_NewPresized(n) PyDict_New()
+#endif
+#if PY_MAJOR_VERSION >= 3 || CYTHON_FUTURE_DIVISION
+ #define __Pyx_PyNumber_Divide(x,y) PyNumber_TrueDivide(x,y)
+ #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceTrueDivide(x,y)
+#else
+ #define __Pyx_PyNumber_Divide(x,y) PyNumber_Divide(x,y)
+ #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceDivide(x,y)
+#endif
+#if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 && CYTHON_USE_UNICODE_INTERNALS
+#define __Pyx_PyDict_GetItemStr(dict, name) _PyDict_GetItem_KnownHash(dict, name, ((PyASCIIObject *) name)->hash)
+#else
+#define __Pyx_PyDict_GetItemStr(dict, name) PyDict_GetItem(dict, name)
+#endif
+#if PY_VERSION_HEX > 0x03030000 && defined(PyUnicode_KIND)
+ #define CYTHON_PEP393_ENABLED 1
+ #define __Pyx_PyUnicode_READY(op) (likely(PyUnicode_IS_READY(op)) ?\
+ 0 : _PyUnicode_Ready((PyObject *)(op)))
+ #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_LENGTH(u)
+ #define __Pyx_PyUnicode_READ_CHAR(u, i) PyUnicode_READ_CHAR(u, i)
+ #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) PyUnicode_MAX_CHAR_VALUE(u)
+ #define __Pyx_PyUnicode_KIND(u) PyUnicode_KIND(u)
+ #define __Pyx_PyUnicode_DATA(u) PyUnicode_DATA(u)
+ #define __Pyx_PyUnicode_READ(k, d, i) PyUnicode_READ(k, d, i)
+ #define __Pyx_PyUnicode_WRITE(k, d, i, ch) PyUnicode_WRITE(k, d, i, ch)
+ #if defined(PyUnicode_IS_READY) && defined(PyUnicode_GET_SIZE)
+ #define __Pyx_PyUnicode_IS_TRUE(u) (0 != (likely(PyUnicode_IS_READY(u)) ? PyUnicode_GET_LENGTH(u) : PyUnicode_GET_SIZE(u)))
+ #else
+ #define __Pyx_PyUnicode_IS_TRUE(u) (0 != PyUnicode_GET_LENGTH(u))
+ #endif
+#else
+ #define CYTHON_PEP393_ENABLED 0
+ #define PyUnicode_1BYTE_KIND 1
+ #define PyUnicode_2BYTE_KIND 2
+ #define PyUnicode_4BYTE_KIND 4
+ #define __Pyx_PyUnicode_READY(op) (0)
+ #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_SIZE(u)
+ #define __Pyx_PyUnicode_READ_CHAR(u, i) ((Py_UCS4)(PyUnicode_AS_UNICODE(u)[i]))
+ #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) ((sizeof(Py_UNICODE) == 2) ? 65535 : 1114111)
+ #define __Pyx_PyUnicode_KIND(u) (sizeof(Py_UNICODE))
+ #define __Pyx_PyUnicode_DATA(u) ((void*)PyUnicode_AS_UNICODE(u))
+ #define __Pyx_PyUnicode_READ(k, d, i) ((void)(k), (Py_UCS4)(((Py_UNICODE*)d)[i]))
+ #define __Pyx_PyUnicode_WRITE(k, d, i, ch) (((void)(k)), ((Py_UNICODE*)d)[i] = ch)
+ #define __Pyx_PyUnicode_IS_TRUE(u) (0 != PyUnicode_GET_SIZE(u))
+#endif
+#if CYTHON_COMPILING_IN_PYPY
+ #define __Pyx_PyUnicode_Concat(a, b) PyNumber_Add(a, b)
+ #define __Pyx_PyUnicode_ConcatSafe(a, b) PyNumber_Add(a, b)
+#else
+ #define __Pyx_PyUnicode_Concat(a, b) PyUnicode_Concat(a, b)
+ #define __Pyx_PyUnicode_ConcatSafe(a, b) ((unlikely((a) == Py_None) || unlikely((b) == Py_None)) ?\
+ PyNumber_Add(a, b) : __Pyx_PyUnicode_Concat(a, b))
+#endif
+#if CYTHON_COMPILING_IN_PYPY && !defined(PyUnicode_Contains)
+ #define PyUnicode_Contains(u, s) PySequence_Contains(u, s)
+#endif
+#if CYTHON_COMPILING_IN_PYPY && !defined(PyByteArray_Check)
+ #define PyByteArray_Check(obj) PyObject_TypeCheck(obj, &PyByteArray_Type)
+#endif
+#if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Format)
+ #define PyObject_Format(obj, fmt) PyObject_CallMethod(obj, "__format__", "O", fmt)
+#endif
+#define __Pyx_PyString_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyString_Check(b) && !PyString_CheckExact(b)))) ? PyNumber_Remainder(a, b) : __Pyx_PyString_Format(a, b))
+#define __Pyx_PyUnicode_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyUnicode_Check(b) && !PyUnicode_CheckExact(b)))) ? PyNumber_Remainder(a, b) : PyUnicode_Format(a, b))
+#if PY_MAJOR_VERSION >= 3
+ #define __Pyx_PyString_Format(a, b) PyUnicode_Format(a, b)
+#else
+ #define __Pyx_PyString_Format(a, b) PyString_Format(a, b)
+#endif
+#if PY_MAJOR_VERSION < 3 && !defined(PyObject_ASCII)
+ #define PyObject_ASCII(o) PyObject_Repr(o)
+#endif
+#if PY_MAJOR_VERSION >= 3
+ #define PyBaseString_Type PyUnicode_Type
+ #define PyStringObject PyUnicodeObject
+ #define PyString_Type PyUnicode_Type
+ #define PyString_Check PyUnicode_Check
+ #define PyString_CheckExact PyUnicode_CheckExact
+#ifndef PyObject_Unicode
+ #define PyObject_Unicode PyObject_Str
+#endif
+#endif
+#if PY_MAJOR_VERSION >= 3
+ #define __Pyx_PyBaseString_Check(obj) PyUnicode_Check(obj)
+ #define __Pyx_PyBaseString_CheckExact(obj) PyUnicode_CheckExact(obj)
+#else
+ #define __Pyx_PyBaseString_Check(obj) (PyString_Check(obj) || PyUnicode_Check(obj))
+ #define __Pyx_PyBaseString_CheckExact(obj) (PyString_CheckExact(obj) || PyUnicode_CheckExact(obj))
+#endif
+#ifndef PySet_CheckExact
+ #define PySet_CheckExact(obj) (Py_TYPE(obj) == &PySet_Type)
+#endif
+#if PY_VERSION_HEX >= 0x030900A4
+ #define __Pyx_SET_REFCNT(obj, refcnt) Py_SET_REFCNT(obj, refcnt)
+ #define __Pyx_SET_SIZE(obj, size) Py_SET_SIZE(obj, size)
+#else
+ #define __Pyx_SET_REFCNT(obj, refcnt) Py_REFCNT(obj) = (refcnt)
+ #define __Pyx_SET_SIZE(obj, size) Py_SIZE(obj) = (size)
+#endif
+#if CYTHON_ASSUME_SAFE_MACROS
+ #define __Pyx_PySequence_SIZE(seq) Py_SIZE(seq)
+#else
+ #define __Pyx_PySequence_SIZE(seq) PySequence_Size(seq)
+#endif
+#if PY_MAJOR_VERSION >= 3
+ #define PyIntObject PyLongObject
+ #define PyInt_Type PyLong_Type
+ #define PyInt_Check(op) PyLong_Check(op)
+ #define PyInt_CheckExact(op) PyLong_CheckExact(op)
+ #define PyInt_FromString PyLong_FromString
+ #define PyInt_FromUnicode PyLong_FromUnicode
+ #define PyInt_FromLong PyLong_FromLong
+ #define PyInt_FromSize_t PyLong_FromSize_t
+ #define PyInt_FromSsize_t PyLong_FromSsize_t
+ #define PyInt_AsLong PyLong_AsLong
+ #define PyInt_AS_LONG PyLong_AS_LONG
+ #define PyInt_AsSsize_t PyLong_AsSsize_t
+ #define PyInt_AsUnsignedLongMask PyLong_AsUnsignedLongMask
+ #define PyInt_AsUnsignedLongLongMask PyLong_AsUnsignedLongLongMask
+ #define PyNumber_Int PyNumber_Long
+#endif
+#if PY_MAJOR_VERSION >= 3
+ #define PyBoolObject PyLongObject
+#endif
+#if PY_MAJOR_VERSION >= 3 && CYTHON_COMPILING_IN_PYPY
+ #ifndef PyUnicode_InternFromString
+ #define PyUnicode_InternFromString(s) PyUnicode_FromString(s)
+ #endif
+#endif
+#if PY_VERSION_HEX < 0x030200A4
+ typedef long Py_hash_t;
+ #define __Pyx_PyInt_FromHash_t PyInt_FromLong
+ #define __Pyx_PyInt_AsHash_t PyInt_AsLong
+#else
+ #define __Pyx_PyInt_FromHash_t PyInt_FromSsize_t
+ #define __Pyx_PyInt_AsHash_t PyInt_AsSsize_t
+#endif
+#if PY_MAJOR_VERSION >= 3
+ #define __Pyx_PyMethod_New(func, self, klass) ((self) ? ((void)(klass), PyMethod_New(func, self)) : __Pyx_NewRef(func))
+#else
+ #define __Pyx_PyMethod_New(func, self, klass) PyMethod_New(func, self, klass)
+#endif
+#if CYTHON_USE_ASYNC_SLOTS
+ #if PY_VERSION_HEX >= 0x030500B1
+ #define __Pyx_PyAsyncMethodsStruct PyAsyncMethods
+ #define __Pyx_PyType_AsAsync(obj) (Py_TYPE(obj)->tp_as_async)
+ #else
+ #define __Pyx_PyType_AsAsync(obj) ((__Pyx_PyAsyncMethodsStruct*) (Py_TYPE(obj)->tp_reserved))
+ #endif
+#else
+ #define __Pyx_PyType_AsAsync(obj) NULL
+#endif
+#ifndef __Pyx_PyAsyncMethodsStruct
+ typedef struct {
+ unaryfunc am_await;
+ unaryfunc am_aiter;
+ unaryfunc am_anext;
+ } __Pyx_PyAsyncMethodsStruct;
+#endif
+
+#if defined(WIN32) || defined(MS_WINDOWS)
+ #define _USE_MATH_DEFINES
+#endif
+#include
+#ifdef NAN
+#define __PYX_NAN() ((float) NAN)
+#else
+static CYTHON_INLINE float __PYX_NAN() {
+ float value;
+ memset(&value, 0xFF, sizeof(value));
+ return value;
+}
+#endif
+#if defined(__CYGWIN__) && defined(_LDBL_EQ_DBL)
+#define __Pyx_truncl trunc
+#else
+#define __Pyx_truncl truncl
+#endif
+
+#define __PYX_MARK_ERR_POS(f_index, lineno) \
+ { __pyx_filename = __pyx_f[f_index]; (void)__pyx_filename; __pyx_lineno = lineno; (void)__pyx_lineno; __pyx_clineno = __LINE__; (void)__pyx_clineno; }
+#define __PYX_ERR(f_index, lineno, Ln_error) \
+ { __PYX_MARK_ERR_POS(f_index, lineno) goto Ln_error; }
+
+#ifndef __PYX_EXTERN_C
+ #ifdef __cplusplus
+ #define __PYX_EXTERN_C extern "C"
+ #else
+ #define __PYX_EXTERN_C extern
+ #endif
+#endif
+
+#define __PYX_HAVE__libmr
+#define __PYX_HAVE_API__libmr
+/* Early includes */
+#include "MetaRecognition.h"
+#include
+#include
+#include "ios"
+#include "new"
+#include "stdexcept"
+#include "typeinfo"
+#include
+#include
+#include "numpy/arrayobject.h"
+#include "numpy/ufuncobject.h"
+
+ /* NumPy API declarations from "numpy/__init__.pxd" */
+
+#include "pythread.h"
+#include "pystate.h"
+#ifdef _OPENMP
+#include
+#endif /* _OPENMP */
+
+#if defined(PYREX_WITHOUT_ASSERTIONS) && !defined(CYTHON_WITHOUT_ASSERTIONS)
+#define CYTHON_WITHOUT_ASSERTIONS
+#endif
+
+typedef struct {PyObject **p; const char *s; const Py_ssize_t n; const char* encoding;
+ const char is_unicode; const char is_str; const char intern; } __Pyx_StringTabEntry;
+
+#define __PYX_DEFAULT_STRING_ENCODING_IS_ASCII 0
+#define __PYX_DEFAULT_STRING_ENCODING_IS_UTF8 0
+#define __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT (PY_MAJOR_VERSION >= 3 && __PYX_DEFAULT_STRING_ENCODING_IS_UTF8)
+#define __PYX_DEFAULT_STRING_ENCODING ""
+#define __Pyx_PyObject_FromString __Pyx_PyBytes_FromString
+#define __Pyx_PyObject_FromStringAndSize __Pyx_PyBytes_FromStringAndSize
+#define __Pyx_uchar_cast(c) ((unsigned char)c)
+#define __Pyx_long_cast(x) ((long)x)
+#define __Pyx_fits_Py_ssize_t(v, type, is_signed) (\
+ (sizeof(type) < sizeof(Py_ssize_t)) ||\
+ (sizeof(type) > sizeof(Py_ssize_t) &&\
+ likely(v < (type)PY_SSIZE_T_MAX ||\
+ v == (type)PY_SSIZE_T_MAX) &&\
+ (!is_signed || likely(v > (type)PY_SSIZE_T_MIN ||\
+ v == (type)PY_SSIZE_T_MIN))) ||\
+ (sizeof(type) == sizeof(Py_ssize_t) &&\
+ (is_signed || likely(v < (type)PY_SSIZE_T_MAX ||\
+ v == (type)PY_SSIZE_T_MAX))) )
+static CYTHON_INLINE int __Pyx_is_valid_index(Py_ssize_t i, Py_ssize_t limit) {
+ return (size_t) i < (size_t) limit;
+}
+#if defined (__cplusplus) && __cplusplus >= 201103L
+ #include
+ #define __Pyx_sst_abs(value) std::abs(value)
+#elif SIZEOF_INT >= SIZEOF_SIZE_T
+ #define __Pyx_sst_abs(value) abs(value)
+#elif SIZEOF_LONG >= SIZEOF_SIZE_T
+ #define __Pyx_sst_abs(value) labs(value)
+#elif defined (_MSC_VER)
+ #define __Pyx_sst_abs(value) ((Py_ssize_t)_abs64(value))
+#elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L
+ #define __Pyx_sst_abs(value) llabs(value)
+#elif defined (__GNUC__)
+ #define __Pyx_sst_abs(value) __builtin_llabs(value)
+#else
+ #define __Pyx_sst_abs(value) ((value<0) ? -value : value)
+#endif
+static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject*);
+static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject*, Py_ssize_t* length);
+#define __Pyx_PyByteArray_FromString(s) PyByteArray_FromStringAndSize((const char*)s, strlen((const char*)s))
+#define __Pyx_PyByteArray_FromStringAndSize(s, l) PyByteArray_FromStringAndSize((const char*)s, l)
+#define __Pyx_PyBytes_FromString PyBytes_FromString
+#define __Pyx_PyBytes_FromStringAndSize PyBytes_FromStringAndSize
+static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char*);
+#if PY_MAJOR_VERSION < 3
+ #define __Pyx_PyStr_FromString __Pyx_PyBytes_FromString
+ #define __Pyx_PyStr_FromStringAndSize __Pyx_PyBytes_FromStringAndSize
+#else
+ #define __Pyx_PyStr_FromString __Pyx_PyUnicode_FromString
+ #define __Pyx_PyStr_FromStringAndSize __Pyx_PyUnicode_FromStringAndSize
+#endif
+#define __Pyx_PyBytes_AsWritableString(s) ((char*) PyBytes_AS_STRING(s))
+#define __Pyx_PyBytes_AsWritableSString(s) ((signed char*) PyBytes_AS_STRING(s))
+#define __Pyx_PyBytes_AsWritableUString(s) ((unsigned char*) PyBytes_AS_STRING(s))
+#define __Pyx_PyBytes_AsString(s) ((const char*) PyBytes_AS_STRING(s))
+#define __Pyx_PyBytes_AsSString(s) ((const signed char*) PyBytes_AS_STRING(s))
+#define __Pyx_PyBytes_AsUString(s) ((const unsigned char*) PyBytes_AS_STRING(s))
+#define __Pyx_PyObject_AsWritableString(s) ((char*) __Pyx_PyObject_AsString(s))
+#define __Pyx_PyObject_AsWritableSString(s) ((signed char*) __Pyx_PyObject_AsString(s))
+#define __Pyx_PyObject_AsWritableUString(s) ((unsigned char*) __Pyx_PyObject_AsString(s))
+#define __Pyx_PyObject_AsSString(s) ((const signed char*) __Pyx_PyObject_AsString(s))
+#define __Pyx_PyObject_AsUString(s) ((const unsigned char*) __Pyx_PyObject_AsString(s))
+#define __Pyx_PyObject_FromCString(s) __Pyx_PyObject_FromString((const char*)s)
+#define __Pyx_PyBytes_FromCString(s) __Pyx_PyBytes_FromString((const char*)s)
+#define __Pyx_PyByteArray_FromCString(s) __Pyx_PyByteArray_FromString((const char*)s)
+#define __Pyx_PyStr_FromCString(s) __Pyx_PyStr_FromString((const char*)s)
+#define __Pyx_PyUnicode_FromCString(s) __Pyx_PyUnicode_FromString((const char*)s)
+static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) {
+ const Py_UNICODE *u_end = u;
+ while (*u_end++) ;
+ return (size_t)(u_end - u - 1);
+}
+#define __Pyx_PyUnicode_FromUnicode(u) PyUnicode_FromUnicode(u, __Pyx_Py_UNICODE_strlen(u))
+#define __Pyx_PyUnicode_FromUnicodeAndLength PyUnicode_FromUnicode
+#define __Pyx_PyUnicode_AsUnicode PyUnicode_AsUnicode
+#define __Pyx_NewRef(obj) (Py_INCREF(obj), obj)
+#define __Pyx_Owned_Py_None(b) __Pyx_NewRef(Py_None)
+static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b);
+static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject*);
+static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject*);
+static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x);
+#define __Pyx_PySequence_Tuple(obj)\
+ (likely(PyTuple_CheckExact(obj)) ? __Pyx_NewRef(obj) : PySequence_Tuple(obj))
+static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject*);
+static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t);
+#if CYTHON_ASSUME_SAFE_MACROS
+#define __pyx_PyFloat_AsDouble(x) (PyFloat_CheckExact(x) ? PyFloat_AS_DOUBLE(x) : PyFloat_AsDouble(x))
+#else
+#define __pyx_PyFloat_AsDouble(x) PyFloat_AsDouble(x)
+#endif
+#define __pyx_PyFloat_AsFloat(x) ((float) __pyx_PyFloat_AsDouble(x))
+#if PY_MAJOR_VERSION >= 3
+#define __Pyx_PyNumber_Int(x) (PyLong_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Long(x))
+#else
+#define __Pyx_PyNumber_Int(x) (PyInt_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Int(x))
+#endif
+#define __Pyx_PyNumber_Float(x) (PyFloat_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Float(x))
+#if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII
+static int __Pyx_sys_getdefaultencoding_not_ascii;
+static int __Pyx_init_sys_getdefaultencoding_params(void) {
+ PyObject* sys;
+ PyObject* default_encoding = NULL;
+ PyObject* ascii_chars_u = NULL;
+ PyObject* ascii_chars_b = NULL;
+ const char* default_encoding_c;
+ sys = PyImport_ImportModule("sys");
+ if (!sys) goto bad;
+ default_encoding = PyObject_CallMethod(sys, (char*) "getdefaultencoding", NULL);
+ Py_DECREF(sys);
+ if (!default_encoding) goto bad;
+ default_encoding_c = PyBytes_AsString(default_encoding);
+ if (!default_encoding_c) goto bad;
+ if (strcmp(default_encoding_c, "ascii") == 0) {
+ __Pyx_sys_getdefaultencoding_not_ascii = 0;
+ } else {
+ char ascii_chars[128];
+ int c;
+ for (c = 0; c < 128; c++) {
+ ascii_chars[c] = c;
+ }
+ __Pyx_sys_getdefaultencoding_not_ascii = 1;
+ ascii_chars_u = PyUnicode_DecodeASCII(ascii_chars, 128, NULL);
+ if (!ascii_chars_u) goto bad;
+ ascii_chars_b = PyUnicode_AsEncodedString(ascii_chars_u, default_encoding_c, NULL);
+ if (!ascii_chars_b || !PyBytes_Check(ascii_chars_b) || memcmp(ascii_chars, PyBytes_AS_STRING(ascii_chars_b), 128) != 0) {
+ PyErr_Format(
+ PyExc_ValueError,
+ "This module compiled with c_string_encoding=ascii, but default encoding '%.200s' is not a superset of ascii.",
+ default_encoding_c);
+ goto bad;
+ }
+ Py_DECREF(ascii_chars_u);
+ Py_DECREF(ascii_chars_b);
+ }
+ Py_DECREF(default_encoding);
+ return 0;
+bad:
+ Py_XDECREF(default_encoding);
+ Py_XDECREF(ascii_chars_u);
+ Py_XDECREF(ascii_chars_b);
+ return -1;
+}
+#endif
+#if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT && PY_MAJOR_VERSION >= 3
+#define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_DecodeUTF8(c_str, size, NULL)
+#else
+#define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_Decode(c_str, size, __PYX_DEFAULT_STRING_ENCODING, NULL)
+#if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT
+static char* __PYX_DEFAULT_STRING_ENCODING;
+static int __Pyx_init_sys_getdefaultencoding_params(void) {
+ PyObject* sys;
+ PyObject* default_encoding = NULL;
+ char* default_encoding_c;
+ sys = PyImport_ImportModule("sys");
+ if (!sys) goto bad;
+ default_encoding = PyObject_CallMethod(sys, (char*) (const char*) "getdefaultencoding", NULL);
+ Py_DECREF(sys);
+ if (!default_encoding) goto bad;
+ default_encoding_c = PyBytes_AsString(default_encoding);
+ if (!default_encoding_c) goto bad;
+ __PYX_DEFAULT_STRING_ENCODING = (char*) malloc(strlen(default_encoding_c) + 1);
+ if (!__PYX_DEFAULT_STRING_ENCODING) goto bad;
+ strcpy(__PYX_DEFAULT_STRING_ENCODING, default_encoding_c);
+ Py_DECREF(default_encoding);
+ return 0;
+bad:
+ Py_XDECREF(default_encoding);
+ return -1;
+}
+#endif
+#endif
+
+
+/* Test for GCC > 2.95 */
+#if defined(__GNUC__) && (__GNUC__ > 2 || (__GNUC__ == 2 && (__GNUC_MINOR__ > 95)))
+ #define likely(x) __builtin_expect(!!(x), 1)
+ #define unlikely(x) __builtin_expect(!!(x), 0)
+#else /* !__GNUC__ or GCC < 2.95 */
+ #define likely(x) (x)
+ #define unlikely(x) (x)
+#endif /* __GNUC__ */
+static CYTHON_INLINE void __Pyx_pretend_to_initialize(void* ptr) { (void)ptr; }
+
+static PyObject *__pyx_m = NULL;
+static PyObject *__pyx_d;
+static PyObject *__pyx_b;
+static PyObject *__pyx_cython_runtime = NULL;
+static PyObject *__pyx_empty_tuple;
+static PyObject *__pyx_empty_bytes;
+static PyObject *__pyx_empty_unicode;
+static int __pyx_lineno;
+static int __pyx_clineno = 0;
+static const char * __pyx_cfilenm= __FILE__;
+static const char *__pyx_filename;
+
+/* Header.proto */
+#if !defined(CYTHON_CCOMPLEX)
+ #if defined(__cplusplus)
+ #define CYTHON_CCOMPLEX 1
+ #elif defined(_Complex_I)
+ #define CYTHON_CCOMPLEX 1
+ #else
+ #define CYTHON_CCOMPLEX 0
+ #endif
+#endif
+#if CYTHON_CCOMPLEX
+ #ifdef __cplusplus
+ #include
+ #else
+ #include
+ #endif
+#endif
+#if CYTHON_CCOMPLEX && !defined(__cplusplus) && defined(__sun__) && defined(__GNUC__)
+ #undef _Complex_I
+ #define _Complex_I 1.0fj
+#endif
+
+
+static const char *__pyx_f[] = {
+ "libmr.pyx",
+ "stringsource",
+ "__init__.pxd",
+ "type.pxd",
+};
+/* BufferFormatStructs.proto */
+#define IS_UNSIGNED(type) (((type) -1) > 0)
+struct __Pyx_StructField_;
+#define __PYX_BUF_FLAGS_PACKED_STRUCT (1 << 0)
+typedef struct {
+ const char* name;
+ struct __Pyx_StructField_* fields;
+ size_t size;
+ size_t arraysize[8];
+ int ndim;
+ char typegroup;
+ char is_unsigned;
+ int flags;
+} __Pyx_TypeInfo;
+typedef struct __Pyx_StructField_ {
+ __Pyx_TypeInfo* type;
+ const char* name;
+ size_t offset;
+} __Pyx_StructField;
+typedef struct {
+ __Pyx_StructField* field;
+ size_t parent_offset;
+} __Pyx_BufFmt_StackElem;
+typedef struct {
+ __Pyx_StructField root;
+ __Pyx_BufFmt_StackElem* head;
+ size_t fmt_offset;
+ size_t new_count, enc_count;
+ size_t struct_alignment;
+ int is_complex;
+ char enc_type;
+ char new_packmode;
+ char enc_packmode;
+ char is_valid_array;
+} __Pyx_BufFmt_Context;
+
+/* MemviewSliceStruct.proto */
+struct __pyx_memoryview_obj;
+typedef struct {
+ struct __pyx_memoryview_obj *memview;
+ char *data;
+ Py_ssize_t shape[8];
+ Py_ssize_t strides[8];
+ Py_ssize_t suboffsets[8];
+} __Pyx_memviewslice;
+#define __Pyx_MemoryView_Len(m) (m.shape[0])
+
+/* Atomics.proto */
+#include
+#ifndef CYTHON_ATOMICS
+ #define CYTHON_ATOMICS 1
+#endif
+#define __pyx_atomic_int_type int
+#if CYTHON_ATOMICS && __GNUC__ >= 4 && (__GNUC_MINOR__ > 1 ||\
+ (__GNUC_MINOR__ == 1 && __GNUC_PATCHLEVEL >= 2)) &&\
+ !defined(__i386__)
+ #define __pyx_atomic_incr_aligned(value, lock) __sync_fetch_and_add(value, 1)
+ #define __pyx_atomic_decr_aligned(value, lock) __sync_fetch_and_sub(value, 1)
+ #ifdef __PYX_DEBUG_ATOMICS
+ #warning "Using GNU atomics"
+ #endif
+#elif CYTHON_ATOMICS && defined(_MSC_VER) && 0
+ #include
+ #undef __pyx_atomic_int_type
+ #define __pyx_atomic_int_type LONG
+ #define __pyx_atomic_incr_aligned(value, lock) InterlockedIncrement(value)
+ #define __pyx_atomic_decr_aligned(value, lock) InterlockedDecrement(value)
+ #ifdef __PYX_DEBUG_ATOMICS
+ #pragma message ("Using MSVC atomics")
+ #endif
+#elif CYTHON_ATOMICS && (defined(__ICC) || defined(__INTEL_COMPILER)) && 0
+ #define __pyx_atomic_incr_aligned(value, lock) _InterlockedIncrement(value)
+ #define __pyx_atomic_decr_aligned(value, lock) _InterlockedDecrement(value)
+ #ifdef __PYX_DEBUG_ATOMICS
+ #warning "Using Intel atomics"
+ #endif
+#else
+ #undef CYTHON_ATOMICS
+ #define CYTHON_ATOMICS 0
+ #ifdef __PYX_DEBUG_ATOMICS
+ #warning "Not using atomics"
+ #endif
+#endif
+typedef volatile __pyx_atomic_int_type __pyx_atomic_int;
+#if CYTHON_ATOMICS
+ #define __pyx_add_acquisition_count(memview)\
+ __pyx_atomic_incr_aligned(__pyx_get_slice_count_pointer(memview), memview->lock)
+ #define __pyx_sub_acquisition_count(memview)\
+ __pyx_atomic_decr_aligned(__pyx_get_slice_count_pointer(memview), memview->lock)
+#else
+ #define __pyx_add_acquisition_count(memview)\
+ __pyx_add_acquisition_count_locked(__pyx_get_slice_count_pointer(memview), memview->lock)
+ #define __pyx_sub_acquisition_count(memview)\
+ __pyx_sub_acquisition_count_locked(__pyx_get_slice_count_pointer(memview), memview->lock)
+#endif
+
+/* ForceInitThreads.proto */
+#ifndef __PYX_FORCE_INIT_THREADS
+ #define __PYX_FORCE_INIT_THREADS 0
+#endif
+
+/* NoFastGil.proto */
+#define __Pyx_PyGILState_Ensure PyGILState_Ensure
+#define __Pyx_PyGILState_Release PyGILState_Release
+#define __Pyx_FastGIL_Remember()
+#define __Pyx_FastGIL_Forget()
+#define __Pyx_FastGilFuncInit()
+
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":689
+ * # in Cython to enable them only on the right systems.
+ *
+ * ctypedef npy_int8 int8_t # <<<<<<<<<<<<<<
+ * ctypedef npy_int16 int16_t
+ * ctypedef npy_int32 int32_t
+ */
+typedef npy_int8 __pyx_t_5numpy_int8_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":690
+ *
+ * ctypedef npy_int8 int8_t
+ * ctypedef npy_int16 int16_t # <<<<<<<<<<<<<<
+ * ctypedef npy_int32 int32_t
+ * ctypedef npy_int64 int64_t
+ */
+typedef npy_int16 __pyx_t_5numpy_int16_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":691
+ * ctypedef npy_int8 int8_t
+ * ctypedef npy_int16 int16_t
+ * ctypedef npy_int32 int32_t # <<<<<<<<<<<<<<
+ * ctypedef npy_int64 int64_t
+ * #ctypedef npy_int96 int96_t
+ */
+typedef npy_int32 __pyx_t_5numpy_int32_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":692
+ * ctypedef npy_int16 int16_t
+ * ctypedef npy_int32 int32_t
+ * ctypedef npy_int64 int64_t # <<<<<<<<<<<<<<
+ * #ctypedef npy_int96 int96_t
+ * #ctypedef npy_int128 int128_t
+ */
+typedef npy_int64 __pyx_t_5numpy_int64_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":696
+ * #ctypedef npy_int128 int128_t
+ *
+ * ctypedef npy_uint8 uint8_t # <<<<<<<<<<<<<<
+ * ctypedef npy_uint16 uint16_t
+ * ctypedef npy_uint32 uint32_t
+ */
+typedef npy_uint8 __pyx_t_5numpy_uint8_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":697
+ *
+ * ctypedef npy_uint8 uint8_t
+ * ctypedef npy_uint16 uint16_t # <<<<<<<<<<<<<<
+ * ctypedef npy_uint32 uint32_t
+ * ctypedef npy_uint64 uint64_t
+ */
+typedef npy_uint16 __pyx_t_5numpy_uint16_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":698
+ * ctypedef npy_uint8 uint8_t
+ * ctypedef npy_uint16 uint16_t
+ * ctypedef npy_uint32 uint32_t # <<<<<<<<<<<<<<
+ * ctypedef npy_uint64 uint64_t
+ * #ctypedef npy_uint96 uint96_t
+ */
+typedef npy_uint32 __pyx_t_5numpy_uint32_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":699
+ * ctypedef npy_uint16 uint16_t
+ * ctypedef npy_uint32 uint32_t
+ * ctypedef npy_uint64 uint64_t # <<<<<<<<<<<<<<
+ * #ctypedef npy_uint96 uint96_t
+ * #ctypedef npy_uint128 uint128_t
+ */
+typedef npy_uint64 __pyx_t_5numpy_uint64_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":703
+ * #ctypedef npy_uint128 uint128_t
+ *
+ * ctypedef npy_float32 float32_t # <<<<<<<<<<<<<<
+ * ctypedef npy_float64 float64_t
+ * #ctypedef npy_float80 float80_t
+ */
+typedef npy_float32 __pyx_t_5numpy_float32_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":704
+ *
+ * ctypedef npy_float32 float32_t
+ * ctypedef npy_float64 float64_t # <<<<<<<<<<<<<<
+ * #ctypedef npy_float80 float80_t
+ * #ctypedef npy_float128 float128_t
+ */
+typedef npy_float64 __pyx_t_5numpy_float64_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":713
+ * # The int types are mapped a bit surprising --
+ * # numpy.int corresponds to 'l' and numpy.long to 'q'
+ * ctypedef npy_long int_t # <<<<<<<<<<<<<<
+ * ctypedef npy_longlong long_t
+ * ctypedef npy_longlong longlong_t
+ */
+typedef npy_long __pyx_t_5numpy_int_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":714
+ * # numpy.int corresponds to 'l' and numpy.long to 'q'
+ * ctypedef npy_long int_t
+ * ctypedef npy_longlong long_t # <<<<<<<<<<<<<<
+ * ctypedef npy_longlong longlong_t
+ *
+ */
+typedef npy_longlong __pyx_t_5numpy_long_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":715
+ * ctypedef npy_long int_t
+ * ctypedef npy_longlong long_t
+ * ctypedef npy_longlong longlong_t # <<<<<<<<<<<<<<
+ *
+ * ctypedef npy_ulong uint_t
+ */
+typedef npy_longlong __pyx_t_5numpy_longlong_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":717
+ * ctypedef npy_longlong longlong_t
+ *
+ * ctypedef npy_ulong uint_t # <<<<<<<<<<<<<<
+ * ctypedef npy_ulonglong ulong_t
+ * ctypedef npy_ulonglong ulonglong_t
+ */
+typedef npy_ulong __pyx_t_5numpy_uint_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":718
+ *
+ * ctypedef npy_ulong uint_t
+ * ctypedef npy_ulonglong ulong_t # <<<<<<<<<<<<<<
+ * ctypedef npy_ulonglong ulonglong_t
+ *
+ */
+typedef npy_ulonglong __pyx_t_5numpy_ulong_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":719
+ * ctypedef npy_ulong uint_t
+ * ctypedef npy_ulonglong ulong_t
+ * ctypedef npy_ulonglong ulonglong_t # <<<<<<<<<<<<<<
+ *
+ * ctypedef npy_intp intp_t
+ */
+typedef npy_ulonglong __pyx_t_5numpy_ulonglong_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":721
+ * ctypedef npy_ulonglong ulonglong_t
+ *
+ * ctypedef npy_intp intp_t # <<<<<<<<<<<<<<
+ * ctypedef npy_uintp uintp_t
+ *
+ */
+typedef npy_intp __pyx_t_5numpy_intp_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":722
+ *
+ * ctypedef npy_intp intp_t
+ * ctypedef npy_uintp uintp_t # <<<<<<<<<<<<<<
+ *
+ * ctypedef npy_double float_t
+ */
+typedef npy_uintp __pyx_t_5numpy_uintp_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":724
+ * ctypedef npy_uintp uintp_t
+ *
+ * ctypedef npy_double float_t # <<<<<<<<<<<<<<
+ * ctypedef npy_double double_t
+ * ctypedef npy_longdouble longdouble_t
+ */
+typedef npy_double __pyx_t_5numpy_float_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":725
+ *
+ * ctypedef npy_double float_t
+ * ctypedef npy_double double_t # <<<<<<<<<<<<<<
+ * ctypedef npy_longdouble longdouble_t
+ *
+ */
+typedef npy_double __pyx_t_5numpy_double_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":726
+ * ctypedef npy_double float_t
+ * ctypedef npy_double double_t
+ * ctypedef npy_longdouble longdouble_t # <<<<<<<<<<<<<<
+ *
+ * ctypedef npy_cfloat cfloat_t
+ */
+typedef npy_longdouble __pyx_t_5numpy_longdouble_t;
+/* Declarations.proto */
+#if CYTHON_CCOMPLEX
+ #ifdef __cplusplus
+ typedef ::std::complex< float > __pyx_t_float_complex;
+ #else
+ typedef float _Complex __pyx_t_float_complex;
+ #endif
+#else
+ typedef struct { float real, imag; } __pyx_t_float_complex;
+#endif
+static CYTHON_INLINE __pyx_t_float_complex __pyx_t_float_complex_from_parts(float, float);
+
+/* Declarations.proto */
+#if CYTHON_CCOMPLEX
+ #ifdef __cplusplus
+ typedef ::std::complex< double > __pyx_t_double_complex;
+ #else
+ typedef double _Complex __pyx_t_double_complex;
+ #endif
+#else
+ typedef struct { double real, imag; } __pyx_t_double_complex;
+#endif
+static CYTHON_INLINE __pyx_t_double_complex __pyx_t_double_complex_from_parts(double, double);
+
+
+/*--- Type declarations ---*/
+struct __pyx_obj_5libmr_MR;
+struct __pyx_array_obj;
+struct __pyx_MemviewEnum_obj;
+struct __pyx_memoryview_obj;
+struct __pyx_memoryviewslice_obj;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":728
+ * ctypedef npy_longdouble longdouble_t
+ *
+ * ctypedef npy_cfloat cfloat_t # <<<<<<<<<<<<<<
+ * ctypedef npy_cdouble cdouble_t
+ * ctypedef npy_clongdouble clongdouble_t
+ */
+typedef npy_cfloat __pyx_t_5numpy_cfloat_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":729
+ *
+ * ctypedef npy_cfloat cfloat_t
+ * ctypedef npy_cdouble cdouble_t # <<<<<<<<<<<<<<
+ * ctypedef npy_clongdouble clongdouble_t
+ *
+ */
+typedef npy_cdouble __pyx_t_5numpy_cdouble_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":730
+ * ctypedef npy_cfloat cfloat_t
+ * ctypedef npy_cdouble cdouble_t
+ * ctypedef npy_clongdouble clongdouble_t # <<<<<<<<<<<<<<
+ *
+ * ctypedef npy_cdouble complex_t
+ */
+typedef npy_clongdouble __pyx_t_5numpy_clongdouble_t;
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":732
+ * ctypedef npy_clongdouble clongdouble_t
+ *
+ * ctypedef npy_cdouble complex_t # <<<<<<<<<<<<<<
+ *
+ * cdef inline object PyArray_MultiIterNew1(a):
+ */
+typedef npy_cdouble __pyx_t_5numpy_complex_t;
+
+/* "libmr.pyx":95
+ *
+ * # This is the Python wrapper class.
+ * cdef class MR: # <<<<<<<<<<<<<<
+ * cdef MetaRecognition *thisptr
+ * def __cinit__(self, int scores_to_drop=0,
+ */
+struct __pyx_obj_5libmr_MR {
+ PyObject_HEAD
+ MetaRecognition *thisptr;
+};
+
+
+/* "View.MemoryView":105
+ *
+ * @cname("__pyx_array")
+ * cdef class array: # <<<<<<<<<<<<<<
+ *
+ * cdef:
+ */
+struct __pyx_array_obj {
+ PyObject_HEAD
+ struct __pyx_vtabstruct_array *__pyx_vtab;
+ char *data;
+ Py_ssize_t len;
+ char *format;
+ int ndim;
+ Py_ssize_t *_shape;
+ Py_ssize_t *_strides;
+ Py_ssize_t itemsize;
+ PyObject *mode;
+ PyObject *_format;
+ void (*callback_free_data)(void *);
+ int free_data;
+ int dtype_is_object;
+};
+
+
+/* "View.MemoryView":279
+ *
+ * @cname('__pyx_MemviewEnum')
+ * cdef class Enum(object): # <<<<<<<<<<<<<<
+ * cdef object name
+ * def __init__(self, name):
+ */
+struct __pyx_MemviewEnum_obj {
+ PyObject_HEAD
+ PyObject *name;
+};
+
+
+/* "View.MemoryView":330
+ *
+ * @cname('__pyx_memoryview')
+ * cdef class memoryview(object): # <<<<<<<<<<<<<<
+ *
+ * cdef object obj
+ */
+struct __pyx_memoryview_obj {
+ PyObject_HEAD
+ struct __pyx_vtabstruct_memoryview *__pyx_vtab;
+ PyObject *obj;
+ PyObject *_size;
+ PyObject *_array_interface;
+ PyThread_type_lock lock;
+ __pyx_atomic_int acquisition_count[2];
+ __pyx_atomic_int *acquisition_count_aligned_p;
+ Py_buffer view;
+ int flags;
+ int dtype_is_object;
+ __Pyx_TypeInfo *typeinfo;
+};
+
+
+/* "View.MemoryView":965
+ *
+ * @cname('__pyx_memoryviewslice')
+ * cdef class _memoryviewslice(memoryview): # <<<<<<<<<<<<<<
+ * "Internal class for passing memoryview slices to Python"
+ *
+ */
+struct __pyx_memoryviewslice_obj {
+ struct __pyx_memoryview_obj __pyx_base;
+ __Pyx_memviewslice from_slice;
+ PyObject *from_object;
+ PyObject *(*to_object_func)(char *);
+ int (*to_dtype_func)(char *, PyObject *);
+};
+
+
+
+/* "View.MemoryView":105
+ *
+ * @cname("__pyx_array")
+ * cdef class array: # <<<<<<<<<<<<<<
+ *
+ * cdef:
+ */
+
+struct __pyx_vtabstruct_array {
+ PyObject *(*get_memview)(struct __pyx_array_obj *);
+};
+static struct __pyx_vtabstruct_array *__pyx_vtabptr_array;
+
+
+/* "View.MemoryView":330
+ *
+ * @cname('__pyx_memoryview')
+ * cdef class memoryview(object): # <<<<<<<<<<<<<<
+ *
+ * cdef object obj
+ */
+
+struct __pyx_vtabstruct_memoryview {
+ char *(*get_item_pointer)(struct __pyx_memoryview_obj *, PyObject *);
+ PyObject *(*is_slice)(struct __pyx_memoryview_obj *, PyObject *);
+ PyObject *(*setitem_slice_assignment)(struct __pyx_memoryview_obj *, PyObject *, PyObject *);
+ PyObject *(*setitem_slice_assign_scalar)(struct __pyx_memoryview_obj *, struct __pyx_memoryview_obj *, PyObject *);
+ PyObject *(*setitem_indexed)(struct __pyx_memoryview_obj *, PyObject *, PyObject *);
+ PyObject *(*convert_item_to_object)(struct __pyx_memoryview_obj *, char *);
+ PyObject *(*assign_item_from_object)(struct __pyx_memoryview_obj *, char *, PyObject *);
+};
+static struct __pyx_vtabstruct_memoryview *__pyx_vtabptr_memoryview;
+
+
+/* "View.MemoryView":965
+ *
+ * @cname('__pyx_memoryviewslice')
+ * cdef class _memoryviewslice(memoryview): # <<<<<<<<<<<<<<
+ * "Internal class for passing memoryview slices to Python"
+ *
+ */
+
+struct __pyx_vtabstruct__memoryviewslice {
+ struct __pyx_vtabstruct_memoryview __pyx_base;
+};
+static struct __pyx_vtabstruct__memoryviewslice *__pyx_vtabptr__memoryviewslice;
+
+/* --- Runtime support code (head) --- */
+/* Refnanny.proto */
+#ifndef CYTHON_REFNANNY
+ #define CYTHON_REFNANNY 0
+#endif
+#if CYTHON_REFNANNY
+ typedef struct {
+ void (*INCREF)(void*, PyObject*, int);
+ void (*DECREF)(void*, PyObject*, int);
+ void (*GOTREF)(void*, PyObject*, int);
+ void (*GIVEREF)(void*, PyObject*, int);
+ void* (*SetupContext)(const char*, int, const char*);
+ void (*FinishContext)(void**);
+ } __Pyx_RefNannyAPIStruct;
+ static __Pyx_RefNannyAPIStruct *__Pyx_RefNanny = NULL;
+ static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname);
+ #define __Pyx_RefNannyDeclarations void *__pyx_refnanny = NULL;
+#ifdef WITH_THREAD
+ #define __Pyx_RefNannySetupContext(name, acquire_gil)\
+ if (acquire_gil) {\
+ PyGILState_STATE __pyx_gilstate_save = PyGILState_Ensure();\
+ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\
+ PyGILState_Release(__pyx_gilstate_save);\
+ } else {\
+ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\
+ }
+#else
+ #define __Pyx_RefNannySetupContext(name, acquire_gil)\
+ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__)
+#endif
+ #define __Pyx_RefNannyFinishContext()\
+ __Pyx_RefNanny->FinishContext(&__pyx_refnanny)
+ #define __Pyx_INCREF(r) __Pyx_RefNanny->INCREF(__pyx_refnanny, (PyObject *)(r), __LINE__)
+ #define __Pyx_DECREF(r) __Pyx_RefNanny->DECREF(__pyx_refnanny, (PyObject *)(r), __LINE__)
+ #define __Pyx_GOTREF(r) __Pyx_RefNanny->GOTREF(__pyx_refnanny, (PyObject *)(r), __LINE__)
+ #define __Pyx_GIVEREF(r) __Pyx_RefNanny->GIVEREF(__pyx_refnanny, (PyObject *)(r), __LINE__)
+ #define __Pyx_XINCREF(r) do { if((r) != NULL) {__Pyx_INCREF(r); }} while(0)
+ #define __Pyx_XDECREF(r) do { if((r) != NULL) {__Pyx_DECREF(r); }} while(0)
+ #define __Pyx_XGOTREF(r) do { if((r) != NULL) {__Pyx_GOTREF(r); }} while(0)
+ #define __Pyx_XGIVEREF(r) do { if((r) != NULL) {__Pyx_GIVEREF(r);}} while(0)
+#else
+ #define __Pyx_RefNannyDeclarations
+ #define __Pyx_RefNannySetupContext(name, acquire_gil)
+ #define __Pyx_RefNannyFinishContext()
+ #define __Pyx_INCREF(r) Py_INCREF(r)
+ #define __Pyx_DECREF(r) Py_DECREF(r)
+ #define __Pyx_GOTREF(r)
+ #define __Pyx_GIVEREF(r)
+ #define __Pyx_XINCREF(r) Py_XINCREF(r)
+ #define __Pyx_XDECREF(r) Py_XDECREF(r)
+ #define __Pyx_XGOTREF(r)
+ #define __Pyx_XGIVEREF(r)
+#endif
+#define __Pyx_XDECREF_SET(r, v) do {\
+ PyObject *tmp = (PyObject *) r;\
+ r = v; __Pyx_XDECREF(tmp);\
+ } while (0)
+#define __Pyx_DECREF_SET(r, v) do {\
+ PyObject *tmp = (PyObject *) r;\
+ r = v; __Pyx_DECREF(tmp);\
+ } while (0)
+#define __Pyx_CLEAR(r) do { PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);} while(0)
+#define __Pyx_XCLEAR(r) do { if((r) != NULL) {PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);}} while(0)
+
+/* PyObjectGetAttrStr.proto */
+#if CYTHON_USE_TYPE_SLOTS
+static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name);
+#else
+#define __Pyx_PyObject_GetAttrStr(o,n) PyObject_GetAttr(o,n)
+#endif
+
+/* GetBuiltinName.proto */
+static PyObject *__Pyx_GetBuiltinName(PyObject *name);
+
+/* RaiseDoubleKeywords.proto */
+static void __Pyx_RaiseDoubleKeywordsError(const char* func_name, PyObject* kw_name);
+
+/* ParseKeywords.proto */
+static int __Pyx_ParseOptionalKeywords(PyObject *kwds, PyObject **argnames[],\
+ PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args,\
+ const char* function_name);
+
+/* RaiseArgTupleInvalid.proto */
+static void __Pyx_RaiseArgtupleInvalid(const char* func_name, int exact,
+ Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found);
+
+/* GetItemInt.proto */
+#define __Pyx_GetItemInt(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\
+ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\
+ __Pyx_GetItemInt_Fast(o, (Py_ssize_t)i, is_list, wraparound, boundscheck) :\
+ (is_list ? (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL) :\
+ __Pyx_GetItemInt_Generic(o, to_py_func(i))))
+#define __Pyx_GetItemInt_List(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\
+ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\
+ __Pyx_GetItemInt_List_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\
+ (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL))
+static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i,
+ int wraparound, int boundscheck);
+#define __Pyx_GetItemInt_Tuple(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\
+ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\
+ __Pyx_GetItemInt_Tuple_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\
+ (PyErr_SetString(PyExc_IndexError, "tuple index out of range"), (PyObject*)NULL))
+static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i,
+ int wraparound, int boundscheck);
+static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j);
+static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i,
+ int is_list, int wraparound, int boundscheck);
+
+/* PyCFunctionFastCall.proto */
+#if CYTHON_FAST_PYCCALL
+static CYTHON_INLINE PyObject *__Pyx_PyCFunction_FastCall(PyObject *func, PyObject **args, Py_ssize_t nargs);
+#else
+#define __Pyx_PyCFunction_FastCall(func, args, nargs) (assert(0), NULL)
+#endif
+
+/* PyFunctionFastCall.proto */
+#if CYTHON_FAST_PYCALL
+#define __Pyx_PyFunction_FastCall(func, args, nargs)\
+ __Pyx_PyFunction_FastCallDict((func), (args), (nargs), NULL)
+#if 1 || PY_VERSION_HEX < 0x030600B1
+static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, Py_ssize_t nargs, PyObject *kwargs);
+#else
+#define __Pyx_PyFunction_FastCallDict(func, args, nargs, kwargs) _PyFunction_FastCallDict(func, args, nargs, kwargs)
+#endif
+#define __Pyx_BUILD_ASSERT_EXPR(cond)\
+ (sizeof(char [1 - 2*!(cond)]) - 1)
+#ifndef Py_MEMBER_SIZE
+#define Py_MEMBER_SIZE(type, member) sizeof(((type *)0)->member)
+#endif
+ static size_t __pyx_pyframe_localsplus_offset = 0;
+ #include "frameobject.h"
+ #define __Pxy_PyFrame_Initialize_Offsets()\
+ ((void)__Pyx_BUILD_ASSERT_EXPR(sizeof(PyFrameObject) == offsetof(PyFrameObject, f_localsplus) + Py_MEMBER_SIZE(PyFrameObject, f_localsplus)),\
+ (void)(__pyx_pyframe_localsplus_offset = ((size_t)PyFrame_Type.tp_basicsize) - Py_MEMBER_SIZE(PyFrameObject, f_localsplus)))
+ #define __Pyx_PyFrame_GetLocalsplus(frame)\
+ (assert(__pyx_pyframe_localsplus_offset), (PyObject **)(((char *)(frame)) + __pyx_pyframe_localsplus_offset))
+#endif
+
+/* PyObjectCall.proto */
+#if CYTHON_COMPILING_IN_CPYTHON
+static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw);
+#else
+#define __Pyx_PyObject_Call(func, arg, kw) PyObject_Call(func, arg, kw)
+#endif
+
+/* PyObjectCall2Args.proto */
+static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2);
+
+/* PyObjectCallMethO.proto */
+#if CYTHON_COMPILING_IN_CPYTHON
+static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg);
+#endif
+
+/* PyObjectCallOneArg.proto */
+static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg);
+
+/* DictGetItem.proto */
+#if PY_MAJOR_VERSION >= 3 && !CYTHON_COMPILING_IN_PYPY
+static PyObject *__Pyx_PyDict_GetItem(PyObject *d, PyObject* key);
+#define __Pyx_PyObject_Dict_GetItem(obj, name)\
+ (likely(PyDict_CheckExact(obj)) ?\
+ __Pyx_PyDict_GetItem(obj, name) : PyObject_GetItem(obj, name))
+#else
+#define __Pyx_PyDict_GetItem(d, key) PyObject_GetItem(d, key)
+#define __Pyx_PyObject_Dict_GetItem(obj, name) PyObject_GetItem(obj, name)
+#endif
+
+/* PyIntCompare.proto */
+static CYTHON_INLINE PyObject* __Pyx_PyInt_EqObjC(PyObject *op1, PyObject *op2, long intval, long inplace);
+
+/* ObjectGetItem.proto */
+#if CYTHON_USE_TYPE_SLOTS
+static CYTHON_INLINE PyObject *__Pyx_PyObject_GetItem(PyObject *obj, PyObject* key);
+#else
+#define __Pyx_PyObject_GetItem(obj, key) PyObject_GetItem(obj, key)
+#endif
+
+/* PyDictVersioning.proto */
+#if CYTHON_USE_DICT_VERSIONS && CYTHON_USE_TYPE_SLOTS
+#define __PYX_DICT_VERSION_INIT ((PY_UINT64_T) -1)
+#define __PYX_GET_DICT_VERSION(dict) (((PyDictObject*)(dict))->ma_version_tag)
+#define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var)\
+ (version_var) = __PYX_GET_DICT_VERSION(dict);\
+ (cache_var) = (value);
+#define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) {\
+ static PY_UINT64_T __pyx_dict_version = 0;\
+ static PyObject *__pyx_dict_cached_value = NULL;\
+ if (likely(__PYX_GET_DICT_VERSION(DICT) == __pyx_dict_version)) {\
+ (VAR) = __pyx_dict_cached_value;\
+ } else {\
+ (VAR) = __pyx_dict_cached_value = (LOOKUP);\
+ __pyx_dict_version = __PYX_GET_DICT_VERSION(DICT);\
+ }\
+}
+static CYTHON_INLINE PY_UINT64_T __Pyx_get_tp_dict_version(PyObject *obj);
+static CYTHON_INLINE PY_UINT64_T __Pyx_get_object_dict_version(PyObject *obj);
+static CYTHON_INLINE int __Pyx_object_dict_version_matches(PyObject* obj, PY_UINT64_T tp_dict_version, PY_UINT64_T obj_dict_version);
+#else
+#define __PYX_GET_DICT_VERSION(dict) (0)
+#define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var)
+#define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) (VAR) = (LOOKUP);
+#endif
+
+/* GetModuleGlobalName.proto */
+#if CYTHON_USE_DICT_VERSIONS
+#define __Pyx_GetModuleGlobalName(var, name) {\
+ static PY_UINT64_T __pyx_dict_version = 0;\
+ static PyObject *__pyx_dict_cached_value = NULL;\
+ (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\
+ (likely(__pyx_dict_cached_value) ? __Pyx_NewRef(__pyx_dict_cached_value) : __Pyx_GetBuiltinName(name)) :\
+ __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\
+}
+#define __Pyx_GetModuleGlobalNameUncached(var, name) {\
+ PY_UINT64_T __pyx_dict_version;\
+ PyObject *__pyx_dict_cached_value;\
+ (var) = __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\
+}
+static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value);
+#else
+#define __Pyx_GetModuleGlobalName(var, name) (var) = __Pyx__GetModuleGlobalName(name)
+#define __Pyx_GetModuleGlobalNameUncached(var, name) (var) = __Pyx__GetModuleGlobalName(name)
+static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name);
+#endif
+
+/* ExtTypeTest.proto */
+static CYTHON_INLINE int __Pyx_TypeTest(PyObject *obj, PyTypeObject *type);
+
+/* IsLittleEndian.proto */
+static CYTHON_INLINE int __Pyx_Is_Little_Endian(void);
+
+/* BufferFormatCheck.proto */
+static const char* __Pyx_BufFmt_CheckString(__Pyx_BufFmt_Context* ctx, const char* ts);
+static void __Pyx_BufFmt_Init(__Pyx_BufFmt_Context* ctx,
+ __Pyx_BufFmt_StackElem* stack,
+ __Pyx_TypeInfo* type);
+
+/* BufferGetAndValidate.proto */
+#define __Pyx_GetBufferAndValidate(buf, obj, dtype, flags, nd, cast, stack)\
+ ((obj == Py_None || obj == NULL) ?\
+ (__Pyx_ZeroBuffer(buf), 0) :\
+ __Pyx__GetBufferAndValidate(buf, obj, dtype, flags, nd, cast, stack))
+static int __Pyx__GetBufferAndValidate(Py_buffer* buf, PyObject* obj,
+ __Pyx_TypeInfo* dtype, int flags, int nd, int cast, __Pyx_BufFmt_StackElem* stack);
+static void __Pyx_ZeroBuffer(Py_buffer* buf);
+static CYTHON_INLINE void __Pyx_SafeReleaseBuffer(Py_buffer* info);
+static Py_ssize_t __Pyx_minusones[] = { -1, -1, -1, -1, -1, -1, -1, -1 };
+static Py_ssize_t __Pyx_zeros[] = { 0, 0, 0, 0, 0, 0, 0, 0 };
+
+/* BufferIndexError.proto */
+static void __Pyx_RaiseBufferIndexError(int axis);
+
+#define __Pyx_BufPtrStrided1d(type, buf, i0, s0) (type)((char*)buf + i0 * s0)
+/* PyThreadStateGet.proto */
+#if CYTHON_FAST_THREAD_STATE
+#define __Pyx_PyThreadState_declare PyThreadState *__pyx_tstate;
+#define __Pyx_PyThreadState_assign __pyx_tstate = __Pyx_PyThreadState_Current;
+#define __Pyx_PyErr_Occurred() __pyx_tstate->curexc_type
+#else
+#define __Pyx_PyThreadState_declare
+#define __Pyx_PyThreadState_assign
+#define __Pyx_PyErr_Occurred() PyErr_Occurred()
+#endif
+
+/* PyErrFetchRestore.proto */
+#if CYTHON_FAST_THREAD_STATE
+#define __Pyx_PyErr_Clear() __Pyx_ErrRestore(NULL, NULL, NULL)
+#define __Pyx_ErrRestoreWithState(type, value, tb) __Pyx_ErrRestoreInState(PyThreadState_GET(), type, value, tb)
+#define __Pyx_ErrFetchWithState(type, value, tb) __Pyx_ErrFetchInState(PyThreadState_GET(), type, value, tb)
+#define __Pyx_ErrRestore(type, value, tb) __Pyx_ErrRestoreInState(__pyx_tstate, type, value, tb)
+#define __Pyx_ErrFetch(type, value, tb) __Pyx_ErrFetchInState(__pyx_tstate, type, value, tb)
+static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb);
+static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb);
+#if CYTHON_COMPILING_IN_CPYTHON
+#define __Pyx_PyErr_SetNone(exc) (Py_INCREF(exc), __Pyx_ErrRestore((exc), NULL, NULL))
+#else
+#define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc)
+#endif
+#else
+#define __Pyx_PyErr_Clear() PyErr_Clear()
+#define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc)
+#define __Pyx_ErrRestoreWithState(type, value, tb) PyErr_Restore(type, value, tb)
+#define __Pyx_ErrFetchWithState(type, value, tb) PyErr_Fetch(type, value, tb)
+#define __Pyx_ErrRestoreInState(tstate, type, value, tb) PyErr_Restore(type, value, tb)
+#define __Pyx_ErrFetchInState(tstate, type, value, tb) PyErr_Fetch(type, value, tb)
+#define __Pyx_ErrRestore(type, value, tb) PyErr_Restore(type, value, tb)
+#define __Pyx_ErrFetch(type, value, tb) PyErr_Fetch(type, value, tb)
+#endif
+
+/* MemviewSliceInit.proto */
+#define __Pyx_BUF_MAX_NDIMS %(BUF_MAX_NDIMS)d
+#define __Pyx_MEMVIEW_DIRECT 1
+#define __Pyx_MEMVIEW_PTR 2
+#define __Pyx_MEMVIEW_FULL 4
+#define __Pyx_MEMVIEW_CONTIG 8
+#define __Pyx_MEMVIEW_STRIDED 16
+#define __Pyx_MEMVIEW_FOLLOW 32
+#define __Pyx_IS_C_CONTIG 1
+#define __Pyx_IS_F_CONTIG 2
+static int __Pyx_init_memviewslice(
+ struct __pyx_memoryview_obj *memview,
+ int ndim,
+ __Pyx_memviewslice *memviewslice,
+ int memview_is_new_reference);
+static CYTHON_INLINE int __pyx_add_acquisition_count_locked(
+ __pyx_atomic_int *acquisition_count, PyThread_type_lock lock);
+static CYTHON_INLINE int __pyx_sub_acquisition_count_locked(
+ __pyx_atomic_int *acquisition_count, PyThread_type_lock lock);
+#define __pyx_get_slice_count_pointer(memview) (memview->acquisition_count_aligned_p)
+#define __pyx_get_slice_count(memview) (*__pyx_get_slice_count_pointer(memview))
+#define __PYX_INC_MEMVIEW(slice, have_gil) __Pyx_INC_MEMVIEW(slice, have_gil, __LINE__)
+#define __PYX_XDEC_MEMVIEW(slice, have_gil) __Pyx_XDEC_MEMVIEW(slice, have_gil, __LINE__)
+static CYTHON_INLINE void __Pyx_INC_MEMVIEW(__Pyx_memviewslice *, int, int);
+static CYTHON_INLINE void __Pyx_XDEC_MEMVIEW(__Pyx_memviewslice *, int, int);
+
+/* RaiseException.proto */
+static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, PyObject *cause);
+
+/* ArgTypeTest.proto */
+#define __Pyx_ArgTypeTest(obj, type, none_allowed, name, exact)\
+ ((likely((Py_TYPE(obj) == type) | (none_allowed && (obj == Py_None)))) ? 1 :\
+ __Pyx__ArgTypeTest(obj, type, name, exact))
+static int __Pyx__ArgTypeTest(PyObject *obj, PyTypeObject *type, const char *name, int exact);
+
+/* PyObjectCallNoArg.proto */
+#if CYTHON_COMPILING_IN_CPYTHON
+static CYTHON_INLINE PyObject* __Pyx_PyObject_CallNoArg(PyObject *func);
+#else
+#define __Pyx_PyObject_CallNoArg(func) __Pyx_PyObject_Call(func, __pyx_empty_tuple, NULL)
+#endif
+
+/* GetTopmostException.proto */
+#if CYTHON_USE_EXC_INFO_STACK
+static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate);
+#endif
+
+/* SaveResetException.proto */
+#if CYTHON_FAST_THREAD_STATE
+#define __Pyx_ExceptionSave(type, value, tb) __Pyx__ExceptionSave(__pyx_tstate, type, value, tb)
+static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb);
+#define __Pyx_ExceptionReset(type, value, tb) __Pyx__ExceptionReset(__pyx_tstate, type, value, tb)
+static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb);
+#else
+#define __Pyx_ExceptionSave(type, value, tb) PyErr_GetExcInfo(type, value, tb)
+#define __Pyx_ExceptionReset(type, value, tb) PyErr_SetExcInfo(type, value, tb)
+#endif
+
+/* PyErrExceptionMatches.proto */
+#if CYTHON_FAST_THREAD_STATE
+#define __Pyx_PyErr_ExceptionMatches(err) __Pyx_PyErr_ExceptionMatchesInState(__pyx_tstate, err)
+static CYTHON_INLINE int __Pyx_PyErr_ExceptionMatchesInState(PyThreadState* tstate, PyObject* err);
+#else
+#define __Pyx_PyErr_ExceptionMatches(err) PyErr_ExceptionMatches(err)
+#endif
+
+/* GetException.proto */
+#if CYTHON_FAST_THREAD_STATE
+#define __Pyx_GetException(type, value, tb) __Pyx__GetException(__pyx_tstate, type, value, tb)
+static int __Pyx__GetException(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb);
+#else
+static int __Pyx_GetException(PyObject **type, PyObject **value, PyObject **tb);
+#endif
+
+/* IncludeStringH.proto */
+#include
+
+/* BytesEquals.proto */
+static CYTHON_INLINE int __Pyx_PyBytes_Equals(PyObject* s1, PyObject* s2, int equals);
+
+/* UnicodeEquals.proto */
+static CYTHON_INLINE int __Pyx_PyUnicode_Equals(PyObject* s1, PyObject* s2, int equals);
+
+/* StrEquals.proto */
+#if PY_MAJOR_VERSION >= 3
+#define __Pyx_PyString_Equals __Pyx_PyUnicode_Equals
+#else
+#define __Pyx_PyString_Equals __Pyx_PyBytes_Equals
+#endif
+
+/* None.proto */
+static CYTHON_INLINE Py_ssize_t __Pyx_div_Py_ssize_t(Py_ssize_t, Py_ssize_t);
+
+/* UnaryNegOverflows.proto */
+#define UNARY_NEG_WOULD_OVERFLOW(x)\
+ (((x) < 0) & ((unsigned long)(x) == 0-(unsigned long)(x)))
+
+static CYTHON_UNUSED int __pyx_array_getbuffer(PyObject *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags); /*proto*/
+static PyObject *__pyx_array_get_memview(struct __pyx_array_obj *); /*proto*/
+/* GetAttr.proto */
+static CYTHON_INLINE PyObject *__Pyx_GetAttr(PyObject *, PyObject *);
+
+/* decode_c_string_utf16.proto */
+static CYTHON_INLINE PyObject *__Pyx_PyUnicode_DecodeUTF16(const char *s, Py_ssize_t size, const char *errors) {
+ int byteorder = 0;
+ return PyUnicode_DecodeUTF16(s, size, errors, &byteorder);
+}
+static CYTHON_INLINE PyObject *__Pyx_PyUnicode_DecodeUTF16LE(const char *s, Py_ssize_t size, const char *errors) {
+ int byteorder = -1;
+ return PyUnicode_DecodeUTF16(s, size, errors, &byteorder);
+}
+static CYTHON_INLINE PyObject *__Pyx_PyUnicode_DecodeUTF16BE(const char *s, Py_ssize_t size, const char *errors) {
+ int byteorder = 1;
+ return PyUnicode_DecodeUTF16(s, size, errors, &byteorder);
+}
+
+/* decode_c_string.proto */
+static CYTHON_INLINE PyObject* __Pyx_decode_c_string(
+ const char* cstring, Py_ssize_t start, Py_ssize_t stop,
+ const char* encoding, const char* errors,
+ PyObject* (*decode_func)(const char *s, Py_ssize_t size, const char *errors));
+
+/* GetAttr3.proto */
+static CYTHON_INLINE PyObject *__Pyx_GetAttr3(PyObject *, PyObject *, PyObject *);
+
+/* RaiseTooManyValuesToUnpack.proto */
+static CYTHON_INLINE void __Pyx_RaiseTooManyValuesError(Py_ssize_t expected);
+
+/* RaiseNeedMoreValuesToUnpack.proto */
+static CYTHON_INLINE void __Pyx_RaiseNeedMoreValuesError(Py_ssize_t index);
+
+/* RaiseNoneIterError.proto */
+static CYTHON_INLINE void __Pyx_RaiseNoneNotIterableError(void);
+
+/* SwapException.proto */
+#if CYTHON_FAST_THREAD_STATE
+#define __Pyx_ExceptionSwap(type, value, tb) __Pyx__ExceptionSwap(__pyx_tstate, type, value, tb)
+static CYTHON_INLINE void __Pyx__ExceptionSwap(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb);
+#else
+static CYTHON_INLINE void __Pyx_ExceptionSwap(PyObject **type, PyObject **value, PyObject **tb);
+#endif
+
+/* Import.proto */
+static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level);
+
+/* FastTypeChecks.proto */
+#if CYTHON_COMPILING_IN_CPYTHON
+#define __Pyx_TypeCheck(obj, type) __Pyx_IsSubtype(Py_TYPE(obj), (PyTypeObject *)type)
+static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b);
+static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches(PyObject *err, PyObject *type);
+static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches2(PyObject *err, PyObject *type1, PyObject *type2);
+#else
+#define __Pyx_TypeCheck(obj, type) PyObject_TypeCheck(obj, (PyTypeObject *)type)
+#define __Pyx_PyErr_GivenExceptionMatches(err, type) PyErr_GivenExceptionMatches(err, type)
+#define __Pyx_PyErr_GivenExceptionMatches2(err, type1, type2) (PyErr_GivenExceptionMatches(err, type1) || PyErr_GivenExceptionMatches(err, type2))
+#endif
+#define __Pyx_PyException_Check(obj) __Pyx_TypeCheck(obj, PyExc_Exception)
+
+static CYTHON_UNUSED int __pyx_memoryview_getbuffer(PyObject *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags); /*proto*/
+/* ListCompAppend.proto */
+#if CYTHON_USE_PYLIST_INTERNALS && CYTHON_ASSUME_SAFE_MACROS
+static CYTHON_INLINE int __Pyx_ListComp_Append(PyObject* list, PyObject* x) {
+ PyListObject* L = (PyListObject*) list;
+ Py_ssize_t len = Py_SIZE(list);
+ if (likely(L->allocated > len)) {
+ Py_INCREF(x);
+ PyList_SET_ITEM(list, len, x);
+ __Pyx_SET_SIZE(list, len + 1);
+ return 0;
+ }
+ return PyList_Append(list, x);
+}
+#else
+#define __Pyx_ListComp_Append(L,x) PyList_Append(L,x)
+#endif
+
+/* PyIntBinop.proto */
+#if !CYTHON_COMPILING_IN_PYPY
+static PyObject* __Pyx_PyInt_AddObjC(PyObject *op1, PyObject *op2, long intval, int inplace, int zerodivision_check);
+#else
+#define __Pyx_PyInt_AddObjC(op1, op2, intval, inplace, zerodivision_check)\
+ (inplace ? PyNumber_InPlaceAdd(op1, op2) : PyNumber_Add(op1, op2))
+#endif
+
+/* ListExtend.proto */
+static CYTHON_INLINE int __Pyx_PyList_Extend(PyObject* L, PyObject* v) {
+#if CYTHON_COMPILING_IN_CPYTHON
+ PyObject* none = _PyList_Extend((PyListObject*)L, v);
+ if (unlikely(!none))
+ return -1;
+ Py_DECREF(none);
+ return 0;
+#else
+ return PyList_SetSlice(L, PY_SSIZE_T_MAX, PY_SSIZE_T_MAX, v);
+#endif
+}
+
+/* ListAppend.proto */
+#if CYTHON_USE_PYLIST_INTERNALS && CYTHON_ASSUME_SAFE_MACROS
+static CYTHON_INLINE int __Pyx_PyList_Append(PyObject* list, PyObject* x) {
+ PyListObject* L = (PyListObject*) list;
+ Py_ssize_t len = Py_SIZE(list);
+ if (likely(L->allocated > len) & likely(len > (L->allocated >> 1))) {
+ Py_INCREF(x);
+ PyList_SET_ITEM(list, len, x);
+ __Pyx_SET_SIZE(list, len + 1);
+ return 0;
+ }
+ return PyList_Append(list, x);
+}
+#else
+#define __Pyx_PyList_Append(L,x) PyList_Append(L,x)
+#endif
+
+/* None.proto */
+static CYTHON_INLINE void __Pyx_RaiseUnboundLocalError(const char *varname);
+
+/* None.proto */
+static CYTHON_INLINE long __Pyx_div_long(long, long);
+
+/* ImportFrom.proto */
+static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name);
+
+/* HasAttr.proto */
+static CYTHON_INLINE int __Pyx_HasAttr(PyObject *, PyObject *);
+
+/* PyObject_GenericGetAttrNoDict.proto */
+#if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000
+static CYTHON_INLINE PyObject* __Pyx_PyObject_GenericGetAttrNoDict(PyObject* obj, PyObject* attr_name);
+#else
+#define __Pyx_PyObject_GenericGetAttrNoDict PyObject_GenericGetAttr
+#endif
+
+/* PyObject_GenericGetAttr.proto */
+#if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000
+static PyObject* __Pyx_PyObject_GenericGetAttr(PyObject* obj, PyObject* attr_name);
+#else
+#define __Pyx_PyObject_GenericGetAttr PyObject_GenericGetAttr
+#endif
+
+/* PyObjectGetAttrStrNoError.proto */
+static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStrNoError(PyObject* obj, PyObject* attr_name);
+
+/* SetupReduce.proto */
+static int __Pyx_setup_reduce(PyObject* type_obj);
+
+/* SetVTable.proto */
+static int __Pyx_SetVtable(PyObject *dict, void *vtable);
+
+/* TypeImport.proto */
+#ifndef __PYX_HAVE_RT_ImportType_proto
+#define __PYX_HAVE_RT_ImportType_proto
+enum __Pyx_ImportType_CheckSize {
+ __Pyx_ImportType_CheckSize_Error = 0,
+ __Pyx_ImportType_CheckSize_Warn = 1,
+ __Pyx_ImportType_CheckSize_Ignore = 2
+};
+static PyTypeObject *__Pyx_ImportType(PyObject* module, const char *module_name, const char *class_name, size_t size, enum __Pyx_ImportType_CheckSize check_size);
+#endif
+
+/* CLineInTraceback.proto */
+#ifdef CYTHON_CLINE_IN_TRACEBACK
+#define __Pyx_CLineForTraceback(tstate, c_line) (((CYTHON_CLINE_IN_TRACEBACK)) ? c_line : 0)
+#else
+static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line);
+#endif
+
+/* CodeObjectCache.proto */
+typedef struct {
+ PyCodeObject* code_object;
+ int code_line;
+} __Pyx_CodeObjectCacheEntry;
+struct __Pyx_CodeObjectCache {
+ int count;
+ int max_count;
+ __Pyx_CodeObjectCacheEntry* entries;
+};
+static struct __Pyx_CodeObjectCache __pyx_code_cache = {0,0,NULL};
+static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line);
+static PyCodeObject *__pyx_find_code_object(int code_line);
+static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object);
+
+/* AddTraceback.proto */
+static void __Pyx_AddTraceback(const char *funcname, int c_line,
+ int py_line, const char *filename);
+
+#if PY_MAJOR_VERSION < 3
+ static int __Pyx_GetBuffer(PyObject *obj, Py_buffer *view, int flags);
+ static void __Pyx_ReleaseBuffer(Py_buffer *view);
+#else
+ #define __Pyx_GetBuffer PyObject_GetBuffer
+ #define __Pyx_ReleaseBuffer PyBuffer_Release
+#endif
+
+
+/* BufferStructDeclare.proto */
+typedef struct {
+ Py_ssize_t shape, strides, suboffsets;
+} __Pyx_Buf_DimInfo;
+typedef struct {
+ size_t refcount;
+ Py_buffer pybuffer;
+} __Pyx_Buffer;
+typedef struct {
+ __Pyx_Buffer *rcbuffer;
+ char *data;
+ __Pyx_Buf_DimInfo diminfo[8];
+} __Pyx_LocalBuf_ND;
+
+/* MemviewSliceIsContig.proto */
+static int __pyx_memviewslice_is_contig(const __Pyx_memviewslice mvs, char order, int ndim);
+
+/* OverlappingSlices.proto */
+static int __pyx_slices_overlap(__Pyx_memviewslice *slice1,
+ __Pyx_memviewslice *slice2,
+ int ndim, size_t itemsize);
+
+/* Capsule.proto */
+static CYTHON_INLINE PyObject *__pyx_capsule_create(void *p, const char *sig);
+
+/* GCCDiagnostics.proto */
+#if defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 6))
+#define __Pyx_HAS_GCC_DIAGNOSTIC
+#endif
+
+/* TypeInfoCompare.proto */
+static int __pyx_typeinfo_cmp(__Pyx_TypeInfo *a, __Pyx_TypeInfo *b);
+
+/* MemviewSliceValidateAndInit.proto */
+static int __Pyx_ValidateAndInit_memviewslice(
+ int *axes_specs,
+ int c_or_f_flag,
+ int buf_flags,
+ int ndim,
+ __Pyx_TypeInfo *dtype,
+ __Pyx_BufFmt_StackElem stack[],
+ __Pyx_memviewslice *memviewslice,
+ PyObject *original_obj);
+
+/* ObjectToMemviewSlice.proto */
+static CYTHON_INLINE __Pyx_memviewslice __Pyx_PyObject_to_MemoryviewSlice_dc_double(PyObject *, int writable_flag);
+
+/* CppExceptionConversion.proto */
+#ifndef __Pyx_CppExn2PyErr
+#include
+#include
+#include
+#include
+static void __Pyx_CppExn2PyErr() {
+ try {
+ if (PyErr_Occurred())
+ ; // let the latest Python exn pass through and ignore the current one
+ else
+ throw;
+ } catch (const std::bad_alloc& exn) {
+ PyErr_SetString(PyExc_MemoryError, exn.what());
+ } catch (const std::bad_cast& exn) {
+ PyErr_SetString(PyExc_TypeError, exn.what());
+ } catch (const std::bad_typeid& exn) {
+ PyErr_SetString(PyExc_TypeError, exn.what());
+ } catch (const std::domain_error& exn) {
+ PyErr_SetString(PyExc_ValueError, exn.what());
+ } catch (const std::invalid_argument& exn) {
+ PyErr_SetString(PyExc_ValueError, exn.what());
+ } catch (const std::ios_base::failure& exn) {
+ PyErr_SetString(PyExc_IOError, exn.what());
+ } catch (const std::out_of_range& exn) {
+ PyErr_SetString(PyExc_IndexError, exn.what());
+ } catch (const std::overflow_error& exn) {
+ PyErr_SetString(PyExc_OverflowError, exn.what());
+ } catch (const std::range_error& exn) {
+ PyErr_SetString(PyExc_ArithmeticError, exn.what());
+ } catch (const std::underflow_error& exn) {
+ PyErr_SetString(PyExc_ArithmeticError, exn.what());
+ } catch (const std::exception& exn) {
+ PyErr_SetString(PyExc_RuntimeError, exn.what());
+ }
+ catch (...)
+ {
+ PyErr_SetString(PyExc_RuntimeError, "Unknown exception");
+ }
+}
+#endif
+
+/* Print.proto */
+static int __Pyx_Print(PyObject*, PyObject *, int);
+#if CYTHON_COMPILING_IN_PYPY || PY_MAJOR_VERSION >= 3
+static PyObject* __pyx_print = 0;
+static PyObject* __pyx_print_kwargs = 0;
+#endif
+
+/* MemviewDtypeToObject.proto */
+static CYTHON_INLINE PyObject *__pyx_memview_get_double(const char *itemp);
+static CYTHON_INLINE int __pyx_memview_set_double(const char *itemp, PyObject *obj);
+
+/* RealImag.proto */
+#if CYTHON_CCOMPLEX
+ #ifdef __cplusplus
+ #define __Pyx_CREAL(z) ((z).real())
+ #define __Pyx_CIMAG(z) ((z).imag())
+ #else
+ #define __Pyx_CREAL(z) (__real__(z))
+ #define __Pyx_CIMAG(z) (__imag__(z))
+ #endif
+#else
+ #define __Pyx_CREAL(z) ((z).real)
+ #define __Pyx_CIMAG(z) ((z).imag)
+#endif
+#if defined(__cplusplus) && CYTHON_CCOMPLEX\
+ && (defined(_WIN32) || defined(__clang__) || (defined(__GNUC__) && (__GNUC__ >= 5 || __GNUC__ == 4 && __GNUC_MINOR__ >= 4 )) || __cplusplus >= 201103)
+ #define __Pyx_SET_CREAL(z,x) ((z).real(x))
+ #define __Pyx_SET_CIMAG(z,y) ((z).imag(y))
+#else
+ #define __Pyx_SET_CREAL(z,x) __Pyx_CREAL(z) = (x)
+ #define __Pyx_SET_CIMAG(z,y) __Pyx_CIMAG(z) = (y)
+#endif
+
+/* Arithmetic.proto */
+#if CYTHON_CCOMPLEX
+ #define __Pyx_c_eq_float(a, b) ((a)==(b))
+ #define __Pyx_c_sum_float(a, b) ((a)+(b))
+ #define __Pyx_c_diff_float(a, b) ((a)-(b))
+ #define __Pyx_c_prod_float(a, b) ((a)*(b))
+ #define __Pyx_c_quot_float(a, b) ((a)/(b))
+ #define __Pyx_c_neg_float(a) (-(a))
+ #ifdef __cplusplus
+ #define __Pyx_c_is_zero_float(z) ((z)==(float)0)
+ #define __Pyx_c_conj_float(z) (::std::conj(z))
+ #if 1
+ #define __Pyx_c_abs_float(z) (::std::abs(z))
+ #define __Pyx_c_pow_float(a, b) (::std::pow(a, b))
+ #endif
+ #else
+ #define __Pyx_c_is_zero_float(z) ((z)==0)
+ #define __Pyx_c_conj_float(z) (conjf(z))
+ #if 1
+ #define __Pyx_c_abs_float(z) (cabsf(z))
+ #define __Pyx_c_pow_float(a, b) (cpowf(a, b))
+ #endif
+ #endif
+#else
+ static CYTHON_INLINE int __Pyx_c_eq_float(__pyx_t_float_complex, __pyx_t_float_complex);
+ static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_sum_float(__pyx_t_float_complex, __pyx_t_float_complex);
+ static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_diff_float(__pyx_t_float_complex, __pyx_t_float_complex);
+ static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_prod_float(__pyx_t_float_complex, __pyx_t_float_complex);
+ static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_quot_float(__pyx_t_float_complex, __pyx_t_float_complex);
+ static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_neg_float(__pyx_t_float_complex);
+ static CYTHON_INLINE int __Pyx_c_is_zero_float(__pyx_t_float_complex);
+ static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_conj_float(__pyx_t_float_complex);
+ #if 1
+ static CYTHON_INLINE float __Pyx_c_abs_float(__pyx_t_float_complex);
+ static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_pow_float(__pyx_t_float_complex, __pyx_t_float_complex);
+ #endif
+#endif
+
+/* Arithmetic.proto */
+#if CYTHON_CCOMPLEX
+ #define __Pyx_c_eq_double(a, b) ((a)==(b))
+ #define __Pyx_c_sum_double(a, b) ((a)+(b))
+ #define __Pyx_c_diff_double(a, b) ((a)-(b))
+ #define __Pyx_c_prod_double(a, b) ((a)*(b))
+ #define __Pyx_c_quot_double(a, b) ((a)/(b))
+ #define __Pyx_c_neg_double(a) (-(a))
+ #ifdef __cplusplus
+ #define __Pyx_c_is_zero_double(z) ((z)==(double)0)
+ #define __Pyx_c_conj_double(z) (::std::conj(z))
+ #if 1
+ #define __Pyx_c_abs_double(z) (::std::abs(z))
+ #define __Pyx_c_pow_double(a, b) (::std::pow(a, b))
+ #endif
+ #else
+ #define __Pyx_c_is_zero_double(z) ((z)==0)
+ #define __Pyx_c_conj_double(z) (conj(z))
+ #if 1
+ #define __Pyx_c_abs_double(z) (cabs(z))
+ #define __Pyx_c_pow_double(a, b) (cpow(a, b))
+ #endif
+ #endif
+#else
+ static CYTHON_INLINE int __Pyx_c_eq_double(__pyx_t_double_complex, __pyx_t_double_complex);
+ static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_sum_double(__pyx_t_double_complex, __pyx_t_double_complex);
+ static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_diff_double(__pyx_t_double_complex, __pyx_t_double_complex);
+ static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_prod_double(__pyx_t_double_complex, __pyx_t_double_complex);
+ static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_quot_double(__pyx_t_double_complex, __pyx_t_double_complex);
+ static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_neg_double(__pyx_t_double_complex);
+ static CYTHON_INLINE int __Pyx_c_is_zero_double(__pyx_t_double_complex);
+ static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_conj_double(__pyx_t_double_complex);
+ #if 1
+ static CYTHON_INLINE double __Pyx_c_abs_double(__pyx_t_double_complex);
+ static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_pow_double(__pyx_t_double_complex, __pyx_t_double_complex);
+ #endif
+#endif
+
+/* MemviewSliceCopyTemplate.proto */
+static __Pyx_memviewslice
+__pyx_memoryview_copy_new_contig(const __Pyx_memviewslice *from_mvs,
+ const char *mode, int ndim,
+ size_t sizeof_dtype, int contig_flag,
+ int dtype_is_object);
+
+/* CIntFromPy.proto */
+static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *);
+
+/* CIntFromPy.proto */
+static CYTHON_INLINE size_t __Pyx_PyInt_As_size_t(PyObject *);
+
+/* CIntToPy.proto */
+static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value);
+
+/* PrintOne.proto */
+static int __Pyx_PrintOne(PyObject* stream, PyObject *o);
+
+/* CIntFromPy.proto */
+static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *);
+
+/* CIntToPy.proto */
+static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value);
+
+/* CIntFromPy.proto */
+static CYTHON_INLINE char __Pyx_PyInt_As_char(PyObject *);
+
+/* CheckBinaryVersion.proto */
+static int __Pyx_check_binary_version(void);
+
+/* InitStrings.proto */
+static int __Pyx_InitStrings(__Pyx_StringTabEntry *t);
+
+static PyObject *__pyx_array_get_memview(struct __pyx_array_obj *__pyx_v_self); /* proto*/
+static char *__pyx_memoryview_get_item_pointer(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_index); /* proto*/
+static PyObject *__pyx_memoryview_is_slice(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_obj); /* proto*/
+static PyObject *__pyx_memoryview_setitem_slice_assignment(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_dst, PyObject *__pyx_v_src); /* proto*/
+static PyObject *__pyx_memoryview_setitem_slice_assign_scalar(struct __pyx_memoryview_obj *__pyx_v_self, struct __pyx_memoryview_obj *__pyx_v_dst, PyObject *__pyx_v_value); /* proto*/
+static PyObject *__pyx_memoryview_setitem_indexed(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_index, PyObject *__pyx_v_value); /* proto*/
+static PyObject *__pyx_memoryview_convert_item_to_object(struct __pyx_memoryview_obj *__pyx_v_self, char *__pyx_v_itemp); /* proto*/
+static PyObject *__pyx_memoryview_assign_item_from_object(struct __pyx_memoryview_obj *__pyx_v_self, char *__pyx_v_itemp, PyObject *__pyx_v_value); /* proto*/
+static PyObject *__pyx_memoryviewslice_convert_item_to_object(struct __pyx_memoryviewslice_obj *__pyx_v_self, char *__pyx_v_itemp); /* proto*/
+static PyObject *__pyx_memoryviewslice_assign_item_from_object(struct __pyx_memoryviewslice_obj *__pyx_v_self, char *__pyx_v_itemp, PyObject *__pyx_v_value); /* proto*/
+
+/* Module declarations from 'libc.string' */
+
+/* Module declarations from 'libc.stdlib' */
+
+/* Module declarations from 'libcpp' */
+
+/* Module declarations from 'libcpp.string' */
+
+/* Module declarations from 'cpython.buffer' */
+
+/* Module declarations from 'libc.stdio' */
+
+/* Module declarations from '__builtin__' */
+
+/* Module declarations from 'cpython.type' */
+static PyTypeObject *__pyx_ptype_7cpython_4type_type = 0;
+
+/* Module declarations from 'cpython' */
+
+/* Module declarations from 'cpython.object' */
+
+/* Module declarations from 'cpython.ref' */
+
+/* Module declarations from 'cpython.mem' */
+
+/* Module declarations from 'numpy' */
+
+/* Module declarations from 'numpy' */
+static PyTypeObject *__pyx_ptype_5numpy_dtype = 0;
+static PyTypeObject *__pyx_ptype_5numpy_flatiter = 0;
+static PyTypeObject *__pyx_ptype_5numpy_broadcast = 0;
+static PyTypeObject *__pyx_ptype_5numpy_ndarray = 0;
+static PyTypeObject *__pyx_ptype_5numpy_ufunc = 0;
+
+/* Module declarations from 'libmr' */
+static PyTypeObject *__pyx_ptype_5libmr_MR = 0;
+static PyTypeObject *__pyx_array_type = 0;
+static PyTypeObject *__pyx_MemviewEnum_type = 0;
+static PyTypeObject *__pyx_memoryview_type = 0;
+static PyTypeObject *__pyx_memoryviewslice_type = 0;
+static PyObject *generic = 0;
+static PyObject *strided = 0;
+static PyObject *indirect = 0;
+static PyObject *contiguous = 0;
+static PyObject *indirect_contiguous = 0;
+static int __pyx_memoryview_thread_locks_used;
+static PyThread_type_lock __pyx_memoryview_thread_locks[8];
+static CYTHON_INLINE PyObject *__pyx_convert_PyObject_string_to_py_std__in_string(std::string const &); /*proto*/
+static CYTHON_INLINE PyObject *__pyx_convert_PyUnicode_string_to_py_std__in_string(std::string const &); /*proto*/
+static CYTHON_INLINE PyObject *__pyx_convert_PyStr_string_to_py_std__in_string(std::string const &); /*proto*/
+static CYTHON_INLINE PyObject *__pyx_convert_PyBytes_string_to_py_std__in_string(std::string const &); /*proto*/
+static CYTHON_INLINE PyObject *__pyx_convert_PyByteArray_string_to_py_std__in_string(std::string const &); /*proto*/
+static std::string __pyx_convert_string_from_py_std__in_string(PyObject *); /*proto*/
+static struct __pyx_array_obj *__pyx_array_new(PyObject *, Py_ssize_t, char *, char *, char *); /*proto*/
+static void *__pyx_align_pointer(void *, size_t); /*proto*/
+static PyObject *__pyx_memoryview_new(PyObject *, int, int, __Pyx_TypeInfo *); /*proto*/
+static CYTHON_INLINE int __pyx_memoryview_check(PyObject *); /*proto*/
+static PyObject *_unellipsify(PyObject *, int); /*proto*/
+static PyObject *assert_direct_dimensions(Py_ssize_t *, int); /*proto*/
+static struct __pyx_memoryview_obj *__pyx_memview_slice(struct __pyx_memoryview_obj *, PyObject *); /*proto*/
+static int __pyx_memoryview_slice_memviewslice(__Pyx_memviewslice *, Py_ssize_t, Py_ssize_t, Py_ssize_t, int, int, int *, Py_ssize_t, Py_ssize_t, Py_ssize_t, int, int, int, int); /*proto*/
+static char *__pyx_pybuffer_index(Py_buffer *, char *, Py_ssize_t, Py_ssize_t); /*proto*/
+static int __pyx_memslice_transpose(__Pyx_memviewslice *); /*proto*/
+static PyObject *__pyx_memoryview_fromslice(__Pyx_memviewslice, int, PyObject *(*)(char *), int (*)(char *, PyObject *), int); /*proto*/
+static __Pyx_memviewslice *__pyx_memoryview_get_slice_from_memoryview(struct __pyx_memoryview_obj *, __Pyx_memviewslice *); /*proto*/
+static void __pyx_memoryview_slice_copy(struct __pyx_memoryview_obj *, __Pyx_memviewslice *); /*proto*/
+static PyObject *__pyx_memoryview_copy_object(struct __pyx_memoryview_obj *); /*proto*/
+static PyObject *__pyx_memoryview_copy_object_from_slice(struct __pyx_memoryview_obj *, __Pyx_memviewslice *); /*proto*/
+static Py_ssize_t abs_py_ssize_t(Py_ssize_t); /*proto*/
+static char __pyx_get_best_slice_order(__Pyx_memviewslice *, int); /*proto*/
+static void _copy_strided_to_strided(char *, Py_ssize_t *, char *, Py_ssize_t *, Py_ssize_t *, Py_ssize_t *, int, size_t); /*proto*/
+static void copy_strided_to_strided(__Pyx_memviewslice *, __Pyx_memviewslice *, int, size_t); /*proto*/
+static Py_ssize_t __pyx_memoryview_slice_get_size(__Pyx_memviewslice *, int); /*proto*/
+static Py_ssize_t __pyx_fill_contig_strides_array(Py_ssize_t *, Py_ssize_t *, Py_ssize_t, int, char); /*proto*/
+static void *__pyx_memoryview_copy_data_to_temp(__Pyx_memviewslice *, __Pyx_memviewslice *, char, int); /*proto*/
+static int __pyx_memoryview_err_extents(int, Py_ssize_t, Py_ssize_t); /*proto*/
+static int __pyx_memoryview_err_dim(PyObject *, char *, int); /*proto*/
+static int __pyx_memoryview_err(PyObject *, char *); /*proto*/
+static int __pyx_memoryview_copy_contents(__Pyx_memviewslice, __Pyx_memviewslice, int, int, int); /*proto*/
+static void __pyx_memoryview_broadcast_leading(__Pyx_memviewslice *, int, int); /*proto*/
+static void __pyx_memoryview_refcount_copying(__Pyx_memviewslice *, int, int, int); /*proto*/
+static void __pyx_memoryview_refcount_objects_in_slice_with_gil(char *, Py_ssize_t *, Py_ssize_t *, int, int); /*proto*/
+static void __pyx_memoryview_refcount_objects_in_slice(char *, Py_ssize_t *, Py_ssize_t *, int, int); /*proto*/
+static void __pyx_memoryview_slice_assign_scalar(__Pyx_memviewslice *, int, size_t, void *, int); /*proto*/
+static void __pyx_memoryview__slice_assign_scalar(char *, Py_ssize_t *, Py_ssize_t *, int, size_t, void *); /*proto*/
+static PyObject *__pyx_unpickle_Enum__set_state(struct __pyx_MemviewEnum_obj *, PyObject *); /*proto*/
+static __Pyx_TypeInfo __Pyx_TypeInfo_nn___pyx_t_5numpy_double_t = { "double_t", NULL, sizeof(__pyx_t_5numpy_double_t), { 0 }, 0, 'R', 0, 0 };
+static __Pyx_TypeInfo __Pyx_TypeInfo_double = { "double", NULL, sizeof(double), { 0 }, 0, 'R', 0, 0 };
+#define __Pyx_MODULE_NAME "libmr"
+extern int __pyx_module_is_main_libmr;
+int __pyx_module_is_main_libmr = 0;
+
+/* Implementation of 'libmr' */
+static PyObject *__pyx_builtin_xrange;
+static PyObject *__pyx_builtin_range;
+static PyObject *__pyx_builtin_TypeError;
+static PyObject *__pyx_builtin_ImportError;
+static PyObject *__pyx_builtin_ValueError;
+static PyObject *__pyx_builtin_MemoryError;
+static PyObject *__pyx_builtin_enumerate;
+static PyObject *__pyx_builtin_Ellipsis;
+static PyObject *__pyx_builtin_id;
+static PyObject *__pyx_builtin_IndexError;
+static const char __pyx_k_O[] = "O";
+static const char __pyx_k_c[] = "c";
+static const char __pyx_k_d[] = "d";
+static const char __pyx_k_x[] = "x";
+static const char __pyx_k_MR[] = "MR";
+static const char __pyx_k_id[] = "id";
+static const char __pyx_k_np[] = "np";
+static const char __pyx_k_end[] = "end";
+static const char __pyx_k_new[] = "__new__";
+static const char __pyx_k_obj[] = "obj";
+static const char __pyx_k_base[] = "base";
+static const char __pyx_k_dict[] = "__dict__";
+static const char __pyx_k_file[] = "file";
+static const char __pyx_k_main[] = "__main__";
+static const char __pyx_k_mode[] = "mode";
+static const char __pyx_k_name[] = "name";
+static const char __pyx_k_ndim[] = "ndim";
+static const char __pyx_k_pack[] = "pack";
+static const char __pyx_k_pymr[] = "pymr";
+static const char __pyx_k_size[] = "size";
+static const char __pyx_k_step[] = "step";
+static const char __pyx_k_stop[] = "stop";
+static const char __pyx_k_test[] = "__test__";
+static const char __pyx_k_ASCII[] = "ASCII";
+static const char __pyx_k_alpha[] = "alpha";
+static const char __pyx_k_class[] = "__class__";
+static const char __pyx_k_dtype[] = "dtype";
+static const char __pyx_k_error[] = "error";
+static const char __pyx_k_flags[] = "flags";
+static const char __pyx_k_input[] = "input";
+static const char __pyx_k_libmr[] = "libmr";
+static const char __pyx_k_numpy[] = "numpy";
+static const char __pyx_k_print[] = "print";
+static const char __pyx_k_range[] = "range";
+static const char __pyx_k_shape[] = "shape";
+static const char __pyx_k_start[] = "start";
+static const char __pyx_k_zeros[] = "zeros";
+static const char __pyx_k_encode[] = "encode";
+static const char __pyx_k_format[] = "format";
+static const char __pyx_k_import[] = "__import__";
+static const char __pyx_k_labels[] = "labels";
+static const char __pyx_k_name_2[] = "__name__";
+static const char __pyx_k_pickle[] = "pickle";
+static const char __pyx_k_reduce[] = "__reduce__";
+static const char __pyx_k_scores[] = "scores";
+static const char __pyx_k_struct[] = "struct";
+static const char __pyx_k_unpack[] = "unpack";
+static const char __pyx_k_update[] = "update";
+static const char __pyx_k_xrange[] = "xrange";
+static const char __pyx_k_fortran[] = "fortran";
+static const char __pyx_k_has_key[] = "has_key";
+static const char __pyx_k_memview[] = "memview";
+static const char __pyx_k_verbose[] = "verbose";
+static const char __pyx_k_Ellipsis[] = "Ellipsis";
+static const char __pyx_k_fit_size[] = "fit_size";
+static const char __pyx_k_fit_type[] = "fit_type";
+static const char __pyx_k_getstate[] = "__getstate__";
+static const char __pyx_k_itemsize[] = "itemsize";
+static const char __pyx_k_pyx_type[] = "__pyx_type";
+static const char __pyx_k_setstate[] = "__setstate__";
+static const char __pyx_k_svm_data[] = "svm_data";
+static const char __pyx_k_TypeError[] = "TypeError";
+static const char __pyx_k_enumerate[] = "enumerate";
+static const char __pyx_k_inputData[] = "inputData";
+static const char __pyx_k_libmr_pyx[] = "libmr.pyx";
+static const char __pyx_k_pyx_state[] = "__pyx_state";
+static const char __pyx_k_reduce_ex[] = "__reduce_ex__";
+static const char __pyx_k_threshold[] = "threshold";
+static const char __pyx_k_IndexError[] = "IndexError";
+static const char __pyx_k_ValueError[] = "ValueError";
+static const char __pyx_k_pyx_result[] = "__pyx_result";
+static const char __pyx_k_pyx_vtable[] = "__pyx_vtable__";
+static const char __pyx_k_ImportError[] = "ImportError";
+static const char __pyx_k_MR_object_r[] = "";
+static const char __pyx_k_MemoryError[] = "MemoryError";
+static const char __pyx_k_PickleError[] = "PickleError";
+static const char __pyx_k_fitting_size[] = "fitting_size";
+static const char __pyx_k_pyx_checksum[] = "__pyx_checksum";
+static const char __pyx_k_stringsource[] = "stringsource";
+static const char __pyx_k_inputDataSize[] = "inputDataSize";
+static const char __pyx_k_pyx_getbuffer[] = "__pyx_getbuffer";
+static const char __pyx_k_reduce_cython[] = "__reduce_cython__";
+static const char __pyx_k_scores_to_drop[] = "scores_to_drop";
+static const char __pyx_k_View_MemoryView[] = "View.MemoryView";
+static const char __pyx_k_allocate_buffer[] = "allocate_buffer";
+static const char __pyx_k_dtype_is_object[] = "dtype_is_object";
+static const char __pyx_k_pyx_PickleError[] = "__pyx_PickleError";
+static const char __pyx_k_setstate_cython[] = "__setstate_cython__";
+static const char __pyx_k_load_from_string[] = "load_from_string";
+static const char __pyx_k_translate_amount[] = "translate_amount";
+static const char __pyx_k_label_of_interest[] = "label_of_interest";
+static const char __pyx_k_pyx_unpickle_Enum[] = "__pyx_unpickle_Enum";
+static const char __pyx_k_cline_in_traceback[] = "cline_in_traceback";
+static const char __pyx_k_strided_and_direct[] = "";
+static const char __pyx_k_strided_and_indirect[] = "";
+static const char __pyx_k_contiguous_and_direct[] = "";
+static const char __pyx_k_MemoryView_of_r_object[] = "";
+static const char __pyx_k_MemoryView_of_r_at_0x_x[] = "";
+static const char __pyx_k_contiguous_and_indirect[] = "";
+static const char __pyx_k_Cannot_index_with_type_s[] = "Cannot index with type '%s'";
+static const char __pyx_k_label_has_positive_score[] = "label_has_positive_score";
+static const char __pyx_k_Invalid_shape_in_axis_d_d[] = "Invalid shape in axis %d: %d.";
+static const char __pyx_k_itemsize_0_for_cython_array[] = "itemsize <= 0 for cython.array";
+static const char __pyx_k_unable_to_allocate_array_data[] = "unable to allocate array data.";
+static const char __pyx_k_strided_and_direct_or_indirect[] = "";
+static const char __pyx_k_numpy_core_multiarray_failed_to[] = "numpy.core.multiarray failed to import";
+static const char __pyx_k_Buffer_view_does_not_expose_stri[] = "Buffer view does not expose strides";
+static const char __pyx_k_Can_only_create_a_buffer_that_is[] = "Can only create a buffer that is contiguous in memory.";
+static const char __pyx_k_Cannot_assign_to_read_only_memor[] = "Cannot assign to read-only memoryview";
+static const char __pyx_k_Cannot_create_writable_memory_vi[] = "Cannot create writable memory view from read-only memoryview";
+static const char __pyx_k_Data_initizalization_complete_No[] = "Data initizalization complete. Now calling C++ code";
+static const char __pyx_k_Empty_shape_tuple_for_cython_arr[] = "Empty shape tuple for cython.array";
+static const char __pyx_k_Incompatible_checksums_s_vs_0xb0[] = "Incompatible checksums (%s vs 0xb068931 = (name))";
+static const char __pyx_k_Indirect_dimensions_not_supporte[] = "Indirect dimensions not supported";
+static const char __pyx_k_Invalid_mode_expected_c_or_fortr[] = "Invalid mode, expected 'c' or 'fortran', got %s";
+static const char __pyx_k_Out_of_bounds_on_buffer_access_a[] = "Out of bounds on buffer access (axis %d)";
+static const char __pyx_k_Unable_to_convert_item_to_object[] = "Unable to convert item to object";
+static const char __pyx_k_got_differing_extents_in_dimensi[] = "got differing extents in dimension %d (got %d and %d)";
+static const char __pyx_k_no_default___reduce___due_to_non[] = "no default __reduce__ due to non-trivial __cinit__";
+static const char __pyx_k_numpy_core_umath_failed_to_impor[] = "numpy.core.umath failed to import";
+static const char __pyx_k_unable_to_allocate_shape_and_str[] = "unable to allocate shape and strides.";
+static PyObject *__pyx_n_s_ASCII;
+static PyObject *__pyx_kp_s_Buffer_view_does_not_expose_stri;
+static PyObject *__pyx_kp_s_Can_only_create_a_buffer_that_is;
+static PyObject *__pyx_kp_s_Cannot_assign_to_read_only_memor;
+static PyObject *__pyx_kp_s_Cannot_create_writable_memory_vi;
+static PyObject *__pyx_kp_s_Cannot_index_with_type_s;
+static PyObject *__pyx_kp_s_Data_initizalization_complete_No;
+static PyObject *__pyx_n_s_Ellipsis;
+static PyObject *__pyx_kp_s_Empty_shape_tuple_for_cython_arr;
+static PyObject *__pyx_n_s_ImportError;
+static PyObject *__pyx_kp_s_Incompatible_checksums_s_vs_0xb0;
+static PyObject *__pyx_n_s_IndexError;
+static PyObject *__pyx_kp_s_Indirect_dimensions_not_supporte;
+static PyObject *__pyx_kp_s_Invalid_mode_expected_c_or_fortr;
+static PyObject *__pyx_kp_s_Invalid_shape_in_axis_d_d;
+static PyObject *__pyx_n_s_MR;
+static PyObject *__pyx_kp_s_MR_object_r;
+static PyObject *__pyx_n_s_MemoryError;
+static PyObject *__pyx_kp_s_MemoryView_of_r_at_0x_x;
+static PyObject *__pyx_kp_s_MemoryView_of_r_object;
+static PyObject *__pyx_n_b_O;
+static PyObject *__pyx_kp_s_Out_of_bounds_on_buffer_access_a;
+static PyObject *__pyx_n_s_PickleError;
+static PyObject *__pyx_n_s_TypeError;
+static PyObject *__pyx_kp_s_Unable_to_convert_item_to_object;
+static PyObject *__pyx_n_s_ValueError;
+static PyObject *__pyx_n_s_View_MemoryView;
+static PyObject *__pyx_n_s_allocate_buffer;
+static PyObject *__pyx_n_s_alpha;
+static PyObject *__pyx_n_s_base;
+static PyObject *__pyx_n_s_c;
+static PyObject *__pyx_n_u_c;
+static PyObject *__pyx_n_s_class;
+static PyObject *__pyx_n_s_cline_in_traceback;
+static PyObject *__pyx_kp_s_contiguous_and_direct;
+static PyObject *__pyx_kp_s_contiguous_and_indirect;
+static PyObject *__pyx_n_s_d;
+static PyObject *__pyx_n_s_dict;
+static PyObject *__pyx_n_s_dtype;
+static PyObject *__pyx_n_s_dtype_is_object;
+static PyObject *__pyx_n_s_encode;
+static PyObject *__pyx_n_s_end;
+static PyObject *__pyx_n_s_enumerate;
+static PyObject *__pyx_n_s_error;
+static PyObject *__pyx_n_s_file;
+static PyObject *__pyx_n_s_fit_size;
+static PyObject *__pyx_n_s_fit_type;
+static PyObject *__pyx_n_s_fitting_size;
+static PyObject *__pyx_n_s_flags;
+static PyObject *__pyx_n_s_format;
+static PyObject *__pyx_n_s_fortran;
+static PyObject *__pyx_n_u_fortran;
+static PyObject *__pyx_n_s_getstate;
+static PyObject *__pyx_kp_s_got_differing_extents_in_dimensi;
+static PyObject *__pyx_n_s_has_key;
+static PyObject *__pyx_n_s_id;
+static PyObject *__pyx_n_s_import;
+static PyObject *__pyx_n_s_input;
+static PyObject *__pyx_n_s_inputData;
+static PyObject *__pyx_n_s_inputDataSize;
+static PyObject *__pyx_n_s_itemsize;
+static PyObject *__pyx_kp_s_itemsize_0_for_cython_array;
+static PyObject *__pyx_n_s_label_has_positive_score;
+static PyObject *__pyx_n_s_label_of_interest;
+static PyObject *__pyx_n_s_labels;
+static PyObject *__pyx_n_s_libmr;
+static PyObject *__pyx_kp_s_libmr_pyx;
+static PyObject *__pyx_n_s_load_from_string;
+static PyObject *__pyx_n_s_main;
+static PyObject *__pyx_n_s_memview;
+static PyObject *__pyx_n_s_mode;
+static PyObject *__pyx_n_s_name;
+static PyObject *__pyx_n_s_name_2;
+static PyObject *__pyx_n_s_ndim;
+static PyObject *__pyx_n_s_new;
+static PyObject *__pyx_kp_s_no_default___reduce___due_to_non;
+static PyObject *__pyx_n_s_np;
+static PyObject *__pyx_n_s_numpy;
+static PyObject *__pyx_kp_s_numpy_core_multiarray_failed_to;
+static PyObject *__pyx_kp_s_numpy_core_umath_failed_to_impor;
+static PyObject *__pyx_n_s_obj;
+static PyObject *__pyx_n_s_pack;
+static PyObject *__pyx_n_s_pickle;
+static PyObject *__pyx_n_s_print;
+static PyObject *__pyx_n_s_pymr;
+static PyObject *__pyx_n_s_pyx_PickleError;
+static PyObject *__pyx_n_s_pyx_checksum;
+static PyObject *__pyx_n_s_pyx_getbuffer;
+static PyObject *__pyx_n_s_pyx_result;
+static PyObject *__pyx_n_s_pyx_state;
+static PyObject *__pyx_n_s_pyx_type;
+static PyObject *__pyx_n_s_pyx_unpickle_Enum;
+static PyObject *__pyx_n_s_pyx_vtable;
+static PyObject *__pyx_n_s_range;
+static PyObject *__pyx_n_s_reduce;
+static PyObject *__pyx_n_s_reduce_cython;
+static PyObject *__pyx_n_s_reduce_ex;
+static PyObject *__pyx_n_s_scores;
+static PyObject *__pyx_n_s_scores_to_drop;
+static PyObject *__pyx_n_s_setstate;
+static PyObject *__pyx_n_s_setstate_cython;
+static PyObject *__pyx_n_s_shape;
+static PyObject *__pyx_n_s_size;
+static PyObject *__pyx_n_s_start;
+static PyObject *__pyx_n_s_step;
+static PyObject *__pyx_n_s_stop;
+static PyObject *__pyx_kp_s_strided_and_direct;
+static PyObject *__pyx_kp_s_strided_and_direct_or_indirect;
+static PyObject *__pyx_kp_s_strided_and_indirect;
+static PyObject *__pyx_kp_s_stringsource;
+static PyObject *__pyx_n_s_struct;
+static PyObject *__pyx_n_s_svm_data;
+static PyObject *__pyx_n_s_test;
+static PyObject *__pyx_n_s_threshold;
+static PyObject *__pyx_n_s_translate_amount;
+static PyObject *__pyx_kp_s_unable_to_allocate_array_data;
+static PyObject *__pyx_kp_s_unable_to_allocate_shape_and_str;
+static PyObject *__pyx_n_s_unpack;
+static PyObject *__pyx_n_s_update;
+static PyObject *__pyx_n_s_verbose;
+static PyObject *__pyx_n_s_x;
+static PyObject *__pyx_n_s_xrange;
+static PyObject *__pyx_n_s_zeros;
+static int __pyx_pf_5libmr_2MR___cinit__(struct __pyx_obj_5libmr_MR *__pyx_v_self, int __pyx_v_scores_to_drop, int __pyx_v_fitting_size, bool __pyx_v_verbose, double __pyx_v_alpha, int __pyx_v_translate_amount); /* proto */
+static void __pyx_pf_5libmr_2MR_2__dealloc__(struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_4fit_low(struct __pyx_obj_5libmr_MR *__pyx_v_self, PyObject *__pyx_v_inputData, int __pyx_v_fit_size); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_6fit_high(struct __pyx_obj_5libmr_MR *__pyx_v_self, PyObject *__pyx_v_inputData, int __pyx_v_fit_size); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_8mr_save(struct __pyx_obj_5libmr_MR *__pyx_v_self, PyObject *__pyx_v_filename); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_10mr_load(struct __pyx_obj_5libmr_MR *__pyx_v_self, PyObject *__pyx_v_filename); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_12fit_svm(struct __pyx_obj_5libmr_MR *__pyx_v_self, PyObject *__pyx_v_svm_data, PyObject *__pyx_v_inputDataSize, PyObject *__pyx_v_label_of_interest, PyObject *__pyx_v_label_has_positive_score, PyObject *__pyx_v_fit_type, PyObject *__pyx_v_fit_size); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_8is_valid___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_14reset(struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_16predict_match(struct __pyx_obj_5libmr_MR *__pyx_v_self, double __pyx_v_x, double __pyx_v_threshold); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_18w_score(struct __pyx_obj_5libmr_MR *__pyx_v_self, double __pyx_v_x); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_20cdf(struct __pyx_obj_5libmr_MR *__pyx_v_self, double __pyx_v_x); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_22inv(struct __pyx_obj_5libmr_MR *__pyx_v_self, double __pyx_v_p); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_24w_score_vector(struct __pyx_obj_5libmr_MR *__pyx_v_self, __Pyx_memviewslice __pyx_v_invec); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_26__str__(struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_28__repr__(struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_8tailsize___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static int __pyx_pf_5libmr_2MR_8tailsize_2__set__(struct __pyx_obj_5libmr_MR *__pyx_v_self, int __pyx_v_nsize); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_16translate_amount___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static int __pyx_pf_5libmr_2MR_16translate_amount_2__set__(struct __pyx_obj_5libmr_MR *__pyx_v_self, int __pyx_v_ntrans); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_4sign___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static int __pyx_pf_5libmr_2MR_4sign_2__set__(struct __pyx_obj_5libmr_MR *__pyx_v_self, int __pyx_v_nsign); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_11small_score___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static int __pyx_pf_5libmr_2MR_11small_score_2__set__(struct __pyx_obj_5libmr_MR *__pyx_v_self, double __pyx_v_nscore); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_7verbose___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static int __pyx_pf_5libmr_2MR_7verbose_2__set__(struct __pyx_obj_5libmr_MR *__pyx_v_self, bool __pyx_v_verbose); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_30__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_5libmr_MR *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_5libmr_2MR_32__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_5libmr_MR *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */
+static PyObject *__pyx_pf_5libmr_load_from_string(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_input); /* proto */
+static int __pyx_array___pyx_pf_15View_dot_MemoryView_5array___cinit__(struct __pyx_array_obj *__pyx_v_self, PyObject *__pyx_v_shape, Py_ssize_t __pyx_v_itemsize, PyObject *__pyx_v_format, PyObject *__pyx_v_mode, int __pyx_v_allocate_buffer); /* proto */
+static int __pyx_array___pyx_pf_15View_dot_MemoryView_5array_2__getbuffer__(struct __pyx_array_obj *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags); /* proto */
+static void __pyx_array___pyx_pf_15View_dot_MemoryView_5array_4__dealloc__(struct __pyx_array_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_5array_7memview___get__(struct __pyx_array_obj *__pyx_v_self); /* proto */
+static Py_ssize_t __pyx_array___pyx_pf_15View_dot_MemoryView_5array_6__len__(struct __pyx_array_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_array___pyx_pf_15View_dot_MemoryView_5array_8__getattr__(struct __pyx_array_obj *__pyx_v_self, PyObject *__pyx_v_attr); /* proto */
+static PyObject *__pyx_array___pyx_pf_15View_dot_MemoryView_5array_10__getitem__(struct __pyx_array_obj *__pyx_v_self, PyObject *__pyx_v_item); /* proto */
+static int __pyx_array___pyx_pf_15View_dot_MemoryView_5array_12__setitem__(struct __pyx_array_obj *__pyx_v_self, PyObject *__pyx_v_item, PyObject *__pyx_v_value); /* proto */
+static PyObject *__pyx_pf___pyx_array___reduce_cython__(CYTHON_UNUSED struct __pyx_array_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf___pyx_array_2__setstate_cython__(CYTHON_UNUSED struct __pyx_array_obj *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */
+static int __pyx_MemviewEnum___pyx_pf_15View_dot_MemoryView_4Enum___init__(struct __pyx_MemviewEnum_obj *__pyx_v_self, PyObject *__pyx_v_name); /* proto */
+static PyObject *__pyx_MemviewEnum___pyx_pf_15View_dot_MemoryView_4Enum_2__repr__(struct __pyx_MemviewEnum_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf___pyx_MemviewEnum___reduce_cython__(struct __pyx_MemviewEnum_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf___pyx_MemviewEnum_2__setstate_cython__(struct __pyx_MemviewEnum_obj *__pyx_v_self, PyObject *__pyx_v___pyx_state); /* proto */
+static int __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview___cinit__(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_obj, int __pyx_v_flags, int __pyx_v_dtype_is_object); /* proto */
+static void __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_2__dealloc__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_4__getitem__(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_index); /* proto */
+static int __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_6__setitem__(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_index, PyObject *__pyx_v_value); /* proto */
+static int __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_8__getbuffer__(struct __pyx_memoryview_obj *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_1T___get__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_4base___get__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_5shape___get__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_7strides___get__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_10suboffsets___get__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_4ndim___get__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_8itemsize___get__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_6nbytes___get__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_4size___get__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static Py_ssize_t __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_10__len__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_12__repr__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_14__str__(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_16is_c_contig(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_18is_f_contig(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_20copy(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_22copy_fortran(struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf___pyx_memoryview___reduce_cython__(CYTHON_UNUSED struct __pyx_memoryview_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf___pyx_memoryview_2__setstate_cython__(CYTHON_UNUSED struct __pyx_memoryview_obj *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */
+static void __pyx_memoryviewslice___pyx_pf_15View_dot_MemoryView_16_memoryviewslice___dealloc__(struct __pyx_memoryviewslice_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView_16_memoryviewslice_4base___get__(struct __pyx_memoryviewslice_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf___pyx_memoryviewslice___reduce_cython__(CYTHON_UNUSED struct __pyx_memoryviewslice_obj *__pyx_v_self); /* proto */
+static PyObject *__pyx_pf___pyx_memoryviewslice_2__setstate_cython__(CYTHON_UNUSED struct __pyx_memoryviewslice_obj *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */
+static PyObject *__pyx_pf_15View_dot_MemoryView___pyx_unpickle_Enum(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v___pyx_type, long __pyx_v___pyx_checksum, PyObject *__pyx_v___pyx_state); /* proto */
+static PyObject *__pyx_tp_new_5libmr_MR(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/
+static PyObject *__pyx_tp_new_array(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/
+static PyObject *__pyx_tp_new_Enum(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/
+static PyObject *__pyx_tp_new_memoryview(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/
+static PyObject *__pyx_tp_new__memoryviewslice(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/
+static PyObject *__pyx_int_0;
+static PyObject *__pyx_int_1;
+static PyObject *__pyx_int_2;
+static PyObject *__pyx_int_3;
+static PyObject *__pyx_int_4;
+static PyObject *__pyx_int_184977713;
+static PyObject *__pyx_int_neg_1;
+static PyObject *__pyx_tuple_;
+static PyObject *__pyx_tuple__2;
+static PyObject *__pyx_tuple__3;
+static PyObject *__pyx_tuple__4;
+static PyObject *__pyx_tuple__5;
+static PyObject *__pyx_tuple__6;
+static PyObject *__pyx_tuple__7;
+static PyObject *__pyx_tuple__8;
+static PyObject *__pyx_tuple__9;
+static PyObject *__pyx_slice__19;
+static PyObject *__pyx_tuple__10;
+static PyObject *__pyx_tuple__11;
+static PyObject *__pyx_tuple__12;
+static PyObject *__pyx_tuple__13;
+static PyObject *__pyx_tuple__14;
+static PyObject *__pyx_tuple__15;
+static PyObject *__pyx_tuple__16;
+static PyObject *__pyx_tuple__17;
+static PyObject *__pyx_tuple__18;
+static PyObject *__pyx_tuple__20;
+static PyObject *__pyx_tuple__21;
+static PyObject *__pyx_tuple__22;
+static PyObject *__pyx_tuple__23;
+static PyObject *__pyx_tuple__25;
+static PyObject *__pyx_tuple__26;
+static PyObject *__pyx_tuple__27;
+static PyObject *__pyx_tuple__28;
+static PyObject *__pyx_tuple__29;
+static PyObject *__pyx_tuple__30;
+static PyObject *__pyx_codeobj__24;
+static PyObject *__pyx_codeobj__31;
+/* Late includes */
+
+/* "libmr.pyx":97
+ * cdef class MR:
+ * cdef MetaRecognition *thisptr
+ * def __cinit__(self, int scores_to_drop=0, # <<<<<<<<<<<<<<
+ * int fitting_size=9,
+ * bool verbose=False,
+ */
+
+/* Python wrapper */
+static int __pyx_pw_5libmr_2MR_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
+static int __pyx_pw_5libmr_2MR_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
+ int __pyx_v_scores_to_drop;
+ int __pyx_v_fitting_size;
+ bool __pyx_v_verbose;
+ double __pyx_v_alpha;
+ int __pyx_v_translate_amount;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__cinit__ (wrapper)", 0);
+ {
+ static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_scores_to_drop,&__pyx_n_s_fitting_size,&__pyx_n_s_verbose,&__pyx_n_s_alpha,&__pyx_n_s_translate_amount,0};
+ PyObject* values[5] = {0,0,0,0,0};
+ if (unlikely(__pyx_kwds)) {
+ Py_ssize_t kw_args;
+ const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
+ switch (pos_args) {
+ case 5: values[4] = PyTuple_GET_ITEM(__pyx_args, 4);
+ CYTHON_FALLTHROUGH;
+ case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3);
+ CYTHON_FALLTHROUGH;
+ case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
+ CYTHON_FALLTHROUGH;
+ case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ CYTHON_FALLTHROUGH;
+ case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ CYTHON_FALLTHROUGH;
+ case 0: break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ kw_args = PyDict_Size(__pyx_kwds);
+ switch (pos_args) {
+ case 0:
+ if (kw_args > 0) {
+ PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_scores_to_drop);
+ if (value) { values[0] = value; kw_args--; }
+ }
+ CYTHON_FALLTHROUGH;
+ case 1:
+ if (kw_args > 0) {
+ PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_fitting_size);
+ if (value) { values[1] = value; kw_args--; }
+ }
+ CYTHON_FALLTHROUGH;
+ case 2:
+ if (kw_args > 0) {
+ PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_verbose);
+ if (value) { values[2] = value; kw_args--; }
+ }
+ CYTHON_FALLTHROUGH;
+ case 3:
+ if (kw_args > 0) {
+ PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_alpha);
+ if (value) { values[3] = value; kw_args--; }
+ }
+ CYTHON_FALLTHROUGH;
+ case 4:
+ if (kw_args > 0) {
+ PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_translate_amount);
+ if (value) { values[4] = value; kw_args--; }
+ }
+ }
+ if (unlikely(kw_args > 0)) {
+ if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "__cinit__") < 0)) __PYX_ERR(0, 97, __pyx_L3_error)
+ }
+ } else {
+ switch (PyTuple_GET_SIZE(__pyx_args)) {
+ case 5: values[4] = PyTuple_GET_ITEM(__pyx_args, 4);
+ CYTHON_FALLTHROUGH;
+ case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3);
+ CYTHON_FALLTHROUGH;
+ case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
+ CYTHON_FALLTHROUGH;
+ case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ CYTHON_FALLTHROUGH;
+ case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ CYTHON_FALLTHROUGH;
+ case 0: break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ }
+ if (values[0]) {
+ __pyx_v_scores_to_drop = __Pyx_PyInt_As_int(values[0]); if (unlikely((__pyx_v_scores_to_drop == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 97, __pyx_L3_error)
+ } else {
+ __pyx_v_scores_to_drop = ((int)0);
+ }
+ if (values[1]) {
+ __pyx_v_fitting_size = __Pyx_PyInt_As_int(values[1]); if (unlikely((__pyx_v_fitting_size == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 98, __pyx_L3_error)
+ } else {
+ __pyx_v_fitting_size = ((int)9);
+ }
+ if (values[2]) {
+ __pyx_v_verbose = __Pyx_PyObject_IsTrue(values[2]); if (unlikely((__pyx_v_verbose == ((bool)-1)) && PyErr_Occurred())) __PYX_ERR(0, 99, __pyx_L3_error)
+ } else {
+
+ /* "libmr.pyx":99
+ * def __cinit__(self, int scores_to_drop=0,
+ * int fitting_size=9,
+ * bool verbose=False, # <<<<<<<<<<<<<<
+ * double alpha=5.0,
+ * int translate_amount=10000):
+ */
+ __pyx_v_verbose = ((bool)0);
+ }
+ if (values[3]) {
+ __pyx_v_alpha = __pyx_PyFloat_AsDouble(values[3]); if (unlikely((__pyx_v_alpha == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 100, __pyx_L3_error)
+ } else {
+ __pyx_v_alpha = ((double)5.0);
+ }
+ if (values[4]) {
+ __pyx_v_translate_amount = __Pyx_PyInt_As_int(values[4]); if (unlikely((__pyx_v_translate_amount == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 101, __pyx_L3_error)
+ } else {
+ __pyx_v_translate_amount = ((int)0x2710);
+ }
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L5_argtuple_error:;
+ __Pyx_RaiseArgtupleInvalid("__cinit__", 0, 0, 5, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 97, __pyx_L3_error)
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return -1;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR___cinit__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), __pyx_v_scores_to_drop, __pyx_v_fitting_size, __pyx_v_verbose, __pyx_v_alpha, __pyx_v_translate_amount);
+
+ /* "libmr.pyx":97
+ * cdef class MR:
+ * cdef MetaRecognition *thisptr
+ * def __cinit__(self, int scores_to_drop=0, # <<<<<<<<<<<<<<
+ * int fitting_size=9,
+ * bool verbose=False,
+ */
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_pf_5libmr_2MR___cinit__(struct __pyx_obj_5libmr_MR *__pyx_v_self, int __pyx_v_scores_to_drop, int __pyx_v_fitting_size, bool __pyx_v_verbose, double __pyx_v_alpha, int __pyx_v_translate_amount) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ MetaRecognition *__pyx_t_1;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__cinit__", 0);
+
+ /* "libmr.pyx":105
+ * Create a new MR object.
+ * """
+ * self.thisptr = new MetaRecognition(scores_to_drop,fitting_size,verbose,alpha,translate_amount) # <<<<<<<<<<<<<<
+ * def __dealloc__(self):
+ * del self.thisptr
+ */
+ try {
+ __pyx_t_1 = new MetaRecognition(__pyx_v_scores_to_drop, __pyx_v_fitting_size, __pyx_v_verbose, __pyx_v_alpha, __pyx_v_translate_amount);
+ } catch(...) {
+ __Pyx_CppExn2PyErr();
+ __PYX_ERR(0, 105, __pyx_L1_error)
+ }
+ __pyx_v_self->thisptr = __pyx_t_1;
+
+ /* "libmr.pyx":97
+ * cdef class MR:
+ * cdef MetaRecognition *thisptr
+ * def __cinit__(self, int scores_to_drop=0, # <<<<<<<<<<<<<<
+ * int fitting_size=9,
+ * bool verbose=False,
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_AddTraceback("libmr.MR.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = -1;
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":106
+ * """
+ * self.thisptr = new MetaRecognition(scores_to_drop,fitting_size,verbose,alpha,translate_amount)
+ * def __dealloc__(self): # <<<<<<<<<<<<<<
+ * del self.thisptr
+ * def fit_low(self, inputData, int fit_size):
+ */
+
+/* Python wrapper */
+static void __pyx_pw_5libmr_2MR_3__dealloc__(PyObject *__pyx_v_self); /*proto*/
+static void __pyx_pw_5libmr_2MR_3__dealloc__(PyObject *__pyx_v_self) {
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__dealloc__ (wrapper)", 0);
+ __pyx_pf_5libmr_2MR_2__dealloc__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+}
+
+static void __pyx_pf_5libmr_2MR_2__dealloc__(struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__dealloc__", 0);
+
+ /* "libmr.pyx":107
+ * self.thisptr = new MetaRecognition(scores_to_drop,fitting_size,verbose,alpha,translate_amount)
+ * def __dealloc__(self):
+ * del self.thisptr # <<<<<<<<<<<<<<
+ * def fit_low(self, inputData, int fit_size):
+ * """Use fit_low if your data is such that is smaller is better. Fits a
+ */
+ delete __pyx_v_self->thisptr;
+
+ /* "libmr.pyx":106
+ * """
+ * self.thisptr = new MetaRecognition(scores_to_drop,fitting_size,verbose,alpha,translate_amount)
+ * def __dealloc__(self): # <<<<<<<<<<<<<<
+ * del self.thisptr
+ * def fit_low(self, inputData, int fit_size):
+ */
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+}
+
+/* "libmr.pyx":108
+ * def __dealloc__(self):
+ * del self.thisptr
+ * def fit_low(self, inputData, int fit_size): # <<<<<<<<<<<<<<
+ * """Use fit_low if your data is such that is smaller is better. Fits a
+ * MR object to the given data. We'll transform it for you
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_5fit_low(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
+static char __pyx_doc_5libmr_2MR_4fit_low[] = "Use fit_low if your data is such that is smaller is better. Fits a\n MR object to the given data. We'll transform it for you\n and keep the transform parameters in the class so later calls\n to W_score or CDF do the right thing.";
+static PyObject *__pyx_pw_5libmr_2MR_5fit_low(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
+ PyObject *__pyx_v_inputData = 0;
+ int __pyx_v_fit_size;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("fit_low (wrapper)", 0);
+ {
+ static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_inputData,&__pyx_n_s_fit_size,0};
+ PyObject* values[2] = {0,0};
+ if (unlikely(__pyx_kwds)) {
+ Py_ssize_t kw_args;
+ const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
+ switch (pos_args) {
+ case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ CYTHON_FALLTHROUGH;
+ case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ CYTHON_FALLTHROUGH;
+ case 0: break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ kw_args = PyDict_Size(__pyx_kwds);
+ switch (pos_args) {
+ case 0:
+ if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_inputData)) != 0)) kw_args--;
+ else goto __pyx_L5_argtuple_error;
+ CYTHON_FALLTHROUGH;
+ case 1:
+ if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_fit_size)) != 0)) kw_args--;
+ else {
+ __Pyx_RaiseArgtupleInvalid("fit_low", 1, 2, 2, 1); __PYX_ERR(0, 108, __pyx_L3_error)
+ }
+ }
+ if (unlikely(kw_args > 0)) {
+ if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "fit_low") < 0)) __PYX_ERR(0, 108, __pyx_L3_error)
+ }
+ } else if (PyTuple_GET_SIZE(__pyx_args) != 2) {
+ goto __pyx_L5_argtuple_error;
+ } else {
+ values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ }
+ __pyx_v_inputData = values[0];
+ __pyx_v_fit_size = __Pyx_PyInt_As_int(values[1]); if (unlikely((__pyx_v_fit_size == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 108, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L5_argtuple_error:;
+ __Pyx_RaiseArgtupleInvalid("fit_low", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 108, __pyx_L3_error)
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.fit_low", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return NULL;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_4fit_low(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), __pyx_v_inputData, __pyx_v_fit_size);
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_4fit_low(struct __pyx_obj_5libmr_MR *__pyx_v_self, PyObject *__pyx_v_inputData, int __pyx_v_fit_size) {
+ double *__pyx_v_data;
+ Py_ssize_t __pyx_v_i;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ Py_ssize_t __pyx_t_1;
+ Py_ssize_t __pyx_t_2;
+ Py_ssize_t __pyx_t_3;
+ PyObject *__pyx_t_4 = NULL;
+ double __pyx_t_5;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("fit_low", 0);
+
+ /* "libmr.pyx":114
+ * to W_score or CDF do the right thing."""
+ * cdef double *data
+ * data = malloc(sizeof(double)*len(inputData)) # <<<<<<<<<<<<<<
+ * for i in xrange(len(inputData)):
+ * data[i] = inputData[i]
+ */
+ __pyx_t_1 = PyObject_Length(__pyx_v_inputData); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 114, __pyx_L1_error)
+ __pyx_v_data = ((double *)malloc(((sizeof(double)) * __pyx_t_1)));
+
+ /* "libmr.pyx":115
+ * cdef double *data
+ * data = malloc(sizeof(double)*len(inputData))
+ * for i in xrange(len(inputData)): # <<<<<<<<<<<<<<
+ * data[i] = inputData[i]
+ * self.thisptr.FitLow(data, len(inputData), fit_size)
+ */
+ __pyx_t_1 = PyObject_Length(__pyx_v_inputData); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 115, __pyx_L1_error)
+ __pyx_t_2 = __pyx_t_1;
+ for (__pyx_t_3 = 0; __pyx_t_3 < __pyx_t_2; __pyx_t_3+=1) {
+ __pyx_v_i = __pyx_t_3;
+
+ /* "libmr.pyx":116
+ * data = malloc(sizeof(double)*len(inputData))
+ * for i in xrange(len(inputData)):
+ * data[i] = inputData[i] # <<<<<<<<<<<<<<
+ * self.thisptr.FitLow(data, len(inputData), fit_size)
+ * free(data)
+ */
+ __pyx_t_4 = __Pyx_GetItemInt(__pyx_v_inputData, __pyx_v_i, Py_ssize_t, 1, PyInt_FromSsize_t, 0, 1, 1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 116, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_5 = __pyx_PyFloat_AsDouble(__pyx_t_4); if (unlikely((__pyx_t_5 == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 116, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+ (__pyx_v_data[__pyx_v_i]) = __pyx_t_5;
+ }
+
+ /* "libmr.pyx":117
+ * for i in xrange(len(inputData)):
+ * data[i] = inputData[i]
+ * self.thisptr.FitLow(data, len(inputData), fit_size) # <<<<<<<<<<<<<<
+ * free(data)
+ * def fit_high(self, inputData, int fit_size):
+ */
+ __pyx_t_1 = PyObject_Length(__pyx_v_inputData); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 117, __pyx_L1_error)
+ (void)(__pyx_v_self->thisptr->FitLow(__pyx_v_data, __pyx_t_1, __pyx_v_fit_size));
+
+ /* "libmr.pyx":118
+ * data[i] = inputData[i]
+ * self.thisptr.FitLow(data, len(inputData), fit_size)
+ * free(data) # <<<<<<<<<<<<<<
+ * def fit_high(self, inputData, int fit_size):
+ * """Use fit_high if your data is such that is larger is better. Fits a
+ */
+ free(__pyx_v_data);
+
+ /* "libmr.pyx":108
+ * def __dealloc__(self):
+ * del self.thisptr
+ * def fit_low(self, inputData, int fit_size): # <<<<<<<<<<<<<<
+ * """Use fit_low if your data is such that is smaller is better. Fits a
+ * MR object to the given data. We'll transform it for you
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_AddTraceback("libmr.MR.fit_low", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":119
+ * self.thisptr.FitLow(data, len(inputData), fit_size)
+ * free(data)
+ * def fit_high(self, inputData, int fit_size): # <<<<<<<<<<<<<<
+ * """Use fit_high if your data is such that is larger is better. Fits a
+ * MR object to the given data. We'll transform it for you
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_7fit_high(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
+static char __pyx_doc_5libmr_2MR_6fit_high[] = "Use fit_high if your data is such that is larger is better. Fits a\n MR object to the given data. We'll transform it for you\n and keep the transform parameters in the class so later calls\n to W_score or CDF do the right thing.\n ";
+static PyObject *__pyx_pw_5libmr_2MR_7fit_high(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
+ PyObject *__pyx_v_inputData = 0;
+ int __pyx_v_fit_size;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("fit_high (wrapper)", 0);
+ {
+ static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_inputData,&__pyx_n_s_fit_size,0};
+ PyObject* values[2] = {0,0};
+ if (unlikely(__pyx_kwds)) {
+ Py_ssize_t kw_args;
+ const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
+ switch (pos_args) {
+ case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ CYTHON_FALLTHROUGH;
+ case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ CYTHON_FALLTHROUGH;
+ case 0: break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ kw_args = PyDict_Size(__pyx_kwds);
+ switch (pos_args) {
+ case 0:
+ if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_inputData)) != 0)) kw_args--;
+ else goto __pyx_L5_argtuple_error;
+ CYTHON_FALLTHROUGH;
+ case 1:
+ if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_fit_size)) != 0)) kw_args--;
+ else {
+ __Pyx_RaiseArgtupleInvalid("fit_high", 1, 2, 2, 1); __PYX_ERR(0, 119, __pyx_L3_error)
+ }
+ }
+ if (unlikely(kw_args > 0)) {
+ if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "fit_high") < 0)) __PYX_ERR(0, 119, __pyx_L3_error)
+ }
+ } else if (PyTuple_GET_SIZE(__pyx_args) != 2) {
+ goto __pyx_L5_argtuple_error;
+ } else {
+ values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ }
+ __pyx_v_inputData = values[0];
+ __pyx_v_fit_size = __Pyx_PyInt_As_int(values[1]); if (unlikely((__pyx_v_fit_size == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 119, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L5_argtuple_error:;
+ __Pyx_RaiseArgtupleInvalid("fit_high", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 119, __pyx_L3_error)
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.fit_high", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return NULL;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_6fit_high(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), __pyx_v_inputData, __pyx_v_fit_size);
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_6fit_high(struct __pyx_obj_5libmr_MR *__pyx_v_self, PyObject *__pyx_v_inputData, int __pyx_v_fit_size) {
+ double *__pyx_v_data;
+ Py_ssize_t __pyx_v_i;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ Py_ssize_t __pyx_t_1;
+ Py_ssize_t __pyx_t_2;
+ Py_ssize_t __pyx_t_3;
+ PyObject *__pyx_t_4 = NULL;
+ double __pyx_t_5;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("fit_high", 0);
+
+ /* "libmr.pyx":126
+ * """
+ * cdef double *data
+ * data = malloc(sizeof(double)*len(inputData)) # <<<<<<<<<<<<<<
+ * for i in xrange(len(inputData)):
+ * data[i] = inputData[i]
+ */
+ __pyx_t_1 = PyObject_Length(__pyx_v_inputData); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 126, __pyx_L1_error)
+ __pyx_v_data = ((double *)malloc(((sizeof(double)) * __pyx_t_1)));
+
+ /* "libmr.pyx":127
+ * cdef double *data
+ * data = malloc(sizeof(double)*len(inputData))
+ * for i in xrange(len(inputData)): # <<<<<<<<<<<<<<
+ * data[i] = inputData[i]
+ * self.thisptr.FitHigh(data, len(inputData), fit_size)
+ */
+ __pyx_t_1 = PyObject_Length(__pyx_v_inputData); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 127, __pyx_L1_error)
+ __pyx_t_2 = __pyx_t_1;
+ for (__pyx_t_3 = 0; __pyx_t_3 < __pyx_t_2; __pyx_t_3+=1) {
+ __pyx_v_i = __pyx_t_3;
+
+ /* "libmr.pyx":128
+ * data = malloc(sizeof(double)*len(inputData))
+ * for i in xrange(len(inputData)):
+ * data[i] = inputData[i] # <<<<<<<<<<<<<<
+ * self.thisptr.FitHigh(data, len(inputData), fit_size)
+ * free(data)
+ */
+ __pyx_t_4 = __Pyx_GetItemInt(__pyx_v_inputData, __pyx_v_i, Py_ssize_t, 1, PyInt_FromSsize_t, 0, 1, 1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 128, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_5 = __pyx_PyFloat_AsDouble(__pyx_t_4); if (unlikely((__pyx_t_5 == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 128, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+ (__pyx_v_data[__pyx_v_i]) = __pyx_t_5;
+ }
+
+ /* "libmr.pyx":129
+ * for i in xrange(len(inputData)):
+ * data[i] = inputData[i]
+ * self.thisptr.FitHigh(data, len(inputData), fit_size) # <<<<<<<<<<<<<<
+ * free(data)
+ *
+ */
+ __pyx_t_1 = PyObject_Length(__pyx_v_inputData); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 129, __pyx_L1_error)
+ (void)(__pyx_v_self->thisptr->FitHigh(__pyx_v_data, __pyx_t_1, __pyx_v_fit_size));
+
+ /* "libmr.pyx":130
+ * data[i] = inputData[i]
+ * self.thisptr.FitHigh(data, len(inputData), fit_size)
+ * free(data) # <<<<<<<<<<<<<<
+ *
+ * def mr_save(self, filename):
+ */
+ free(__pyx_v_data);
+
+ /* "libmr.pyx":119
+ * self.thisptr.FitLow(data, len(inputData), fit_size)
+ * free(data)
+ * def fit_high(self, inputData, int fit_size): # <<<<<<<<<<<<<<
+ * """Use fit_high if your data is such that is larger is better. Fits a
+ * MR object to the given data. We'll transform it for you
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_AddTraceback("libmr.MR.fit_high", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":132
+ * free(data)
+ *
+ * def mr_save(self, filename): # <<<<<<<<<<<<<<
+ * """
+ * save mr object to file
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_9mr_save(PyObject *__pyx_v_self, PyObject *__pyx_v_filename); /*proto*/
+static char __pyx_doc_5libmr_2MR_8mr_save[] = "\n save mr object to file\n ";
+static PyObject *__pyx_pw_5libmr_2MR_9mr_save(PyObject *__pyx_v_self, PyObject *__pyx_v_filename) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("mr_save (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_8mr_save(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((PyObject *)__pyx_v_filename));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_8mr_save(struct __pyx_obj_5libmr_MR *__pyx_v_self, PyObject *__pyx_v_filename) {
+ char *__pyx_v_filetosave;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ char *__pyx_t_1;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("mr_save", 0);
+
+ /* "libmr.pyx":137
+ * """
+ * cdef char *filetosave
+ * filetosave = filename # <<<<<<<<<<<<<<
+ * self.thisptr.Save(filetosave)
+ *
+ */
+ __pyx_t_1 = __Pyx_PyObject_AsWritableString(__pyx_v_filename); if (unlikely((!__pyx_t_1) && PyErr_Occurred())) __PYX_ERR(0, 137, __pyx_L1_error)
+ __pyx_v_filetosave = __pyx_t_1;
+
+ /* "libmr.pyx":138
+ * cdef char *filetosave
+ * filetosave = filename
+ * self.thisptr.Save(filetosave) # <<<<<<<<<<<<<<
+ *
+ * def mr_load(self, filename):
+ */
+ __pyx_v_self->thisptr->Save(__pyx_v_filetosave);
+
+ /* "libmr.pyx":132
+ * free(data)
+ *
+ * def mr_save(self, filename): # <<<<<<<<<<<<<<
+ * """
+ * save mr object to file
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_AddTraceback("libmr.MR.mr_save", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":140
+ * self.thisptr.Save(filetosave)
+ *
+ * def mr_load(self, filename): # <<<<<<<<<<<<<<
+ * """
+ * save mr object to file
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_11mr_load(PyObject *__pyx_v_self, PyObject *__pyx_v_filename); /*proto*/
+static char __pyx_doc_5libmr_2MR_10mr_load[] = "\n save mr object to file\n ";
+static PyObject *__pyx_pw_5libmr_2MR_11mr_load(PyObject *__pyx_v_self, PyObject *__pyx_v_filename) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("mr_load (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_10mr_load(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((PyObject *)__pyx_v_filename));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_10mr_load(struct __pyx_obj_5libmr_MR *__pyx_v_self, PyObject *__pyx_v_filename) {
+ char *__pyx_v_filetosave;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ char *__pyx_t_1;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("mr_load", 0);
+
+ /* "libmr.pyx":145
+ * """
+ * cdef char *filetosave
+ * filetosave = filename # <<<<<<<<<<<<<<
+ * self.thisptr.Load(filetosave)
+ *
+ */
+ __pyx_t_1 = __Pyx_PyObject_AsWritableString(__pyx_v_filename); if (unlikely((!__pyx_t_1) && PyErr_Occurred())) __PYX_ERR(0, 145, __pyx_L1_error)
+ __pyx_v_filetosave = __pyx_t_1;
+
+ /* "libmr.pyx":146
+ * cdef char *filetosave
+ * filetosave = filename
+ * self.thisptr.Load(filetosave) # <<<<<<<<<<<<<<
+ *
+ * def fit_svm(self, svm_data, inputDataSize, label_of_interest,
+ */
+ __pyx_v_self->thisptr->Load(__pyx_v_filetosave);
+
+ /* "libmr.pyx":140
+ * self.thisptr.Save(filetosave)
+ *
+ * def mr_load(self, filename): # <<<<<<<<<<<<<<
+ * """
+ * save mr object to file
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_AddTraceback("libmr.MR.mr_load", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":148
+ * self.thisptr.Load(filetosave)
+ *
+ * def fit_svm(self, svm_data, inputDataSize, label_of_interest, # <<<<<<<<<<<<<<
+ * label_has_positive_score, fit_type, fit_size ):
+ * """
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_13fit_svm(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
+static char __pyx_doc_5libmr_2MR_12fit_svm[] = "\n Input:\n --------\n svm_data: dict containing labels and decision scores. \n eg. svm_data['scores'] = [], svm_data['labels'] = []\n inputDataSize : total no of decision scores\n label_of_interest : eg +1, -1\n label_has_positive_score : bool i.e 0 or 1\n fit_type : complement_reject=1, positive_reject=2, complement_model=3, positive_model=4\n fit_size : size of tail to be used\n\n Output:\n --------\n None\n You can access parameters from weibull fitting using other attributes.\n Loading/Saving of weibull model parameters can be done using load/save methods\n in MR class\n\n ";
+static PyObject *__pyx_pw_5libmr_2MR_13fit_svm(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
+ PyObject *__pyx_v_svm_data = 0;
+ PyObject *__pyx_v_inputDataSize = 0;
+ PyObject *__pyx_v_label_of_interest = 0;
+ PyObject *__pyx_v_label_has_positive_score = 0;
+ PyObject *__pyx_v_fit_type = 0;
+ PyObject *__pyx_v_fit_size = 0;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("fit_svm (wrapper)", 0);
+ {
+ static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_svm_data,&__pyx_n_s_inputDataSize,&__pyx_n_s_label_of_interest,&__pyx_n_s_label_has_positive_score,&__pyx_n_s_fit_type,&__pyx_n_s_fit_size,0};
+ PyObject* values[6] = {0,0,0,0,0,0};
+ if (unlikely(__pyx_kwds)) {
+ Py_ssize_t kw_args;
+ const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
+ switch (pos_args) {
+ case 6: values[5] = PyTuple_GET_ITEM(__pyx_args, 5);
+ CYTHON_FALLTHROUGH;
+ case 5: values[4] = PyTuple_GET_ITEM(__pyx_args, 4);
+ CYTHON_FALLTHROUGH;
+ case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3);
+ CYTHON_FALLTHROUGH;
+ case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
+ CYTHON_FALLTHROUGH;
+ case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ CYTHON_FALLTHROUGH;
+ case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ CYTHON_FALLTHROUGH;
+ case 0: break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ kw_args = PyDict_Size(__pyx_kwds);
+ switch (pos_args) {
+ case 0:
+ if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_svm_data)) != 0)) kw_args--;
+ else goto __pyx_L5_argtuple_error;
+ CYTHON_FALLTHROUGH;
+ case 1:
+ if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_inputDataSize)) != 0)) kw_args--;
+ else {
+ __Pyx_RaiseArgtupleInvalid("fit_svm", 1, 6, 6, 1); __PYX_ERR(0, 148, __pyx_L3_error)
+ }
+ CYTHON_FALLTHROUGH;
+ case 2:
+ if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_label_of_interest)) != 0)) kw_args--;
+ else {
+ __Pyx_RaiseArgtupleInvalid("fit_svm", 1, 6, 6, 2); __PYX_ERR(0, 148, __pyx_L3_error)
+ }
+ CYTHON_FALLTHROUGH;
+ case 3:
+ if (likely((values[3] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_label_has_positive_score)) != 0)) kw_args--;
+ else {
+ __Pyx_RaiseArgtupleInvalid("fit_svm", 1, 6, 6, 3); __PYX_ERR(0, 148, __pyx_L3_error)
+ }
+ CYTHON_FALLTHROUGH;
+ case 4:
+ if (likely((values[4] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_fit_type)) != 0)) kw_args--;
+ else {
+ __Pyx_RaiseArgtupleInvalid("fit_svm", 1, 6, 6, 4); __PYX_ERR(0, 148, __pyx_L3_error)
+ }
+ CYTHON_FALLTHROUGH;
+ case 5:
+ if (likely((values[5] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_fit_size)) != 0)) kw_args--;
+ else {
+ __Pyx_RaiseArgtupleInvalid("fit_svm", 1, 6, 6, 5); __PYX_ERR(0, 148, __pyx_L3_error)
+ }
+ }
+ if (unlikely(kw_args > 0)) {
+ if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "fit_svm") < 0)) __PYX_ERR(0, 148, __pyx_L3_error)
+ }
+ } else if (PyTuple_GET_SIZE(__pyx_args) != 6) {
+ goto __pyx_L5_argtuple_error;
+ } else {
+ values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
+ values[3] = PyTuple_GET_ITEM(__pyx_args, 3);
+ values[4] = PyTuple_GET_ITEM(__pyx_args, 4);
+ values[5] = PyTuple_GET_ITEM(__pyx_args, 5);
+ }
+ __pyx_v_svm_data = values[0];
+ __pyx_v_inputDataSize = values[1];
+ __pyx_v_label_of_interest = values[2];
+ __pyx_v_label_has_positive_score = values[3];
+ __pyx_v_fit_type = values[4];
+ __pyx_v_fit_size = values[5];
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L5_argtuple_error:;
+ __Pyx_RaiseArgtupleInvalid("fit_svm", 1, 6, 6, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 148, __pyx_L3_error)
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.fit_svm", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return NULL;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_12fit_svm(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), __pyx_v_svm_data, __pyx_v_inputDataSize, __pyx_v_label_of_interest, __pyx_v_label_has_positive_score, __pyx_v_fit_type, __pyx_v_fit_size);
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_12fit_svm(struct __pyx_obj_5libmr_MR *__pyx_v_self, PyObject *__pyx_v_svm_data, PyObject *__pyx_v_inputDataSize, PyObject *__pyx_v_label_of_interest, PyObject *__pyx_v_label_has_positive_score, PyObject *__pyx_v_fit_type, PyObject *__pyx_v_fit_size) {
+ struct svm_node_libsvm *__pyx_v_svm_data_to_c;
+ PyObject *__pyx_v_i = NULL;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ size_t __pyx_t_3;
+ PyObject *__pyx_t_4 = NULL;
+ int __pyx_t_5;
+ Py_ssize_t __pyx_t_6;
+ Py_ssize_t __pyx_t_7;
+ int __pyx_t_8;
+ PyObject *(*__pyx_t_9)(PyObject *);
+ int __pyx_t_10;
+ double __pyx_t_11;
+ int __pyx_t_12;
+ bool __pyx_t_13;
+ int __pyx_t_14;
+ int __pyx_t_15;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("fit_svm", 0);
+
+ /* "libmr.pyx":173
+ * cdef svm_node_libsvm *svm_data_to_c
+ *
+ * svm_data_to_c = < svm_node_libsvm* >malloc(inputDataSize * sizeof(svm_node_libsvm) ) # <<<<<<<<<<<<<<
+ *
+ * assert svm_data.has_key("scores")
+ */
+ __pyx_t_1 = __Pyx_PyInt_FromSize_t((sizeof(struct svm_node_libsvm))); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 173, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_2 = PyNumber_Multiply(__pyx_v_inputDataSize, __pyx_t_1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 173, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_t_3 = __Pyx_PyInt_As_size_t(__pyx_t_2); if (unlikely((__pyx_t_3 == (size_t)-1) && PyErr_Occurred())) __PYX_ERR(0, 173, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __pyx_v_svm_data_to_c = ((struct svm_node_libsvm *)malloc(__pyx_t_3));
+
+ /* "libmr.pyx":175
+ * svm_data_to_c = < svm_node_libsvm* >malloc(inputDataSize * sizeof(svm_node_libsvm) )
+ *
+ * assert svm_data.has_key("scores") # <<<<<<<<<<<<<<
+ * assert svm_data.has_key("scores")
+ * assert len(svm_data["scores"]) == len(svm_data["labels"])
+ */
+ #ifndef CYTHON_WITHOUT_ASSERTIONS
+ if (unlikely(!Py_OptimizeFlag)) {
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_v_svm_data, __pyx_n_s_has_key); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 175, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_4 = NULL;
+ if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_1))) {
+ __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_1);
+ if (likely(__pyx_t_4)) {
+ PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1);
+ __Pyx_INCREF(__pyx_t_4);
+ __Pyx_INCREF(function);
+ __Pyx_DECREF_SET(__pyx_t_1, function);
+ }
+ }
+ __pyx_t_2 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_4, __pyx_n_s_scores) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_n_s_scores);
+ __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0;
+ if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 175, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_t_5 = __Pyx_PyObject_IsTrue(__pyx_t_2); if (unlikely(__pyx_t_5 < 0)) __PYX_ERR(0, 175, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ if (unlikely(!__pyx_t_5)) {
+ PyErr_SetNone(PyExc_AssertionError);
+ __PYX_ERR(0, 175, __pyx_L1_error)
+ }
+ }
+ #endif
+
+ /* "libmr.pyx":176
+ *
+ * assert svm_data.has_key("scores")
+ * assert svm_data.has_key("scores") # <<<<<<<<<<<<<<
+ * assert len(svm_data["scores"]) == len(svm_data["labels"])
+ * assert fit_type in [1, 2, 3, 4]
+ */
+ #ifndef CYTHON_WITHOUT_ASSERTIONS
+ if (unlikely(!Py_OptimizeFlag)) {
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_v_svm_data, __pyx_n_s_has_key); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 176, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_4 = NULL;
+ if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_1))) {
+ __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_1);
+ if (likely(__pyx_t_4)) {
+ PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1);
+ __Pyx_INCREF(__pyx_t_4);
+ __Pyx_INCREF(function);
+ __Pyx_DECREF_SET(__pyx_t_1, function);
+ }
+ }
+ __pyx_t_2 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_4, __pyx_n_s_scores) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_n_s_scores);
+ __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0;
+ if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 176, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_t_5 = __Pyx_PyObject_IsTrue(__pyx_t_2); if (unlikely(__pyx_t_5 < 0)) __PYX_ERR(0, 176, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ if (unlikely(!__pyx_t_5)) {
+ PyErr_SetNone(PyExc_AssertionError);
+ __PYX_ERR(0, 176, __pyx_L1_error)
+ }
+ }
+ #endif
+
+ /* "libmr.pyx":177
+ * assert svm_data.has_key("scores")
+ * assert svm_data.has_key("scores")
+ * assert len(svm_data["scores"]) == len(svm_data["labels"]) # <<<<<<<<<<<<<<
+ * assert fit_type in [1, 2, 3, 4]
+ * for i in range(inputDataSize):
+ */
+ #ifndef CYTHON_WITHOUT_ASSERTIONS
+ if (unlikely(!Py_OptimizeFlag)) {
+ __pyx_t_2 = __Pyx_PyObject_Dict_GetItem(__pyx_v_svm_data, __pyx_n_s_scores); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 177, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_6 = PyObject_Length(__pyx_t_2); if (unlikely(__pyx_t_6 == ((Py_ssize_t)-1))) __PYX_ERR(0, 177, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __pyx_t_2 = __Pyx_PyObject_Dict_GetItem(__pyx_v_svm_data, __pyx_n_s_labels); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 177, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_7 = PyObject_Length(__pyx_t_2); if (unlikely(__pyx_t_7 == ((Py_ssize_t)-1))) __PYX_ERR(0, 177, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ if (unlikely(!((__pyx_t_6 == __pyx_t_7) != 0))) {
+ PyErr_SetNone(PyExc_AssertionError);
+ __PYX_ERR(0, 177, __pyx_L1_error)
+ }
+ }
+ #endif
+
+ /* "libmr.pyx":178
+ * assert svm_data.has_key("scores")
+ * assert len(svm_data["scores"]) == len(svm_data["labels"])
+ * assert fit_type in [1, 2, 3, 4] # <<<<<<<<<<<<<<
+ * for i in range(inputDataSize):
+ * svm_data_to_c[i].index = svm_data["labels"][i]
+ */
+ #ifndef CYTHON_WITHOUT_ASSERTIONS
+ if (unlikely(!Py_OptimizeFlag)) {
+ __Pyx_INCREF(__pyx_v_fit_type);
+ __pyx_t_2 = __pyx_v_fit_type;
+ __pyx_t_1 = __Pyx_PyInt_EqObjC(__pyx_t_2, __pyx_int_1, 1, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 178, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_8 = __Pyx_PyObject_IsTrue(__pyx_t_1); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 178, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ if (!__pyx_t_8) {
+ } else {
+ __pyx_t_5 = __pyx_t_8;
+ goto __pyx_L3_bool_binop_done;
+ }
+ __pyx_t_1 = __Pyx_PyInt_EqObjC(__pyx_t_2, __pyx_int_2, 2, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 178, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_8 = __Pyx_PyObject_IsTrue(__pyx_t_1); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 178, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ if (!__pyx_t_8) {
+ } else {
+ __pyx_t_5 = __pyx_t_8;
+ goto __pyx_L3_bool_binop_done;
+ }
+ __pyx_t_1 = __Pyx_PyInt_EqObjC(__pyx_t_2, __pyx_int_3, 3, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 178, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_8 = __Pyx_PyObject_IsTrue(__pyx_t_1); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 178, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ if (!__pyx_t_8) {
+ } else {
+ __pyx_t_5 = __pyx_t_8;
+ goto __pyx_L3_bool_binop_done;
+ }
+ __pyx_t_1 = __Pyx_PyInt_EqObjC(__pyx_t_2, __pyx_int_4, 4, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 178, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_8 = __Pyx_PyObject_IsTrue(__pyx_t_1); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 178, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_t_5 = __pyx_t_8;
+ __pyx_L3_bool_binop_done:;
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ if (unlikely(!(__pyx_t_5 != 0))) {
+ PyErr_SetNone(PyExc_AssertionError);
+ __PYX_ERR(0, 178, __pyx_L1_error)
+ }
+ }
+ #endif
+
+ /* "libmr.pyx":179
+ * assert len(svm_data["scores"]) == len(svm_data["labels"])
+ * assert fit_type in [1, 2, 3, 4]
+ * for i in range(inputDataSize): # <<<<<<<<<<<<<<
+ * svm_data_to_c[i].index = svm_data["labels"][i]
+ * svm_data_to_c[i].value = svm_data["scores"][i]
+ */
+ __pyx_t_2 = __Pyx_PyObject_CallOneArg(__pyx_builtin_range, __pyx_v_inputDataSize); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 179, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ if (likely(PyList_CheckExact(__pyx_t_2)) || PyTuple_CheckExact(__pyx_t_2)) {
+ __pyx_t_1 = __pyx_t_2; __Pyx_INCREF(__pyx_t_1); __pyx_t_7 = 0;
+ __pyx_t_9 = NULL;
+ } else {
+ __pyx_t_7 = -1; __pyx_t_1 = PyObject_GetIter(__pyx_t_2); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 179, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_9 = Py_TYPE(__pyx_t_1)->tp_iternext; if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 179, __pyx_L1_error)
+ }
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ for (;;) {
+ if (likely(!__pyx_t_9)) {
+ if (likely(PyList_CheckExact(__pyx_t_1))) {
+ if (__pyx_t_7 >= PyList_GET_SIZE(__pyx_t_1)) break;
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_2 = PyList_GET_ITEM(__pyx_t_1, __pyx_t_7); __Pyx_INCREF(__pyx_t_2); __pyx_t_7++; if (unlikely(0 < 0)) __PYX_ERR(0, 179, __pyx_L1_error)
+ #else
+ __pyx_t_2 = PySequence_ITEM(__pyx_t_1, __pyx_t_7); __pyx_t_7++; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 179, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ #endif
+ } else {
+ if (__pyx_t_7 >= PyTuple_GET_SIZE(__pyx_t_1)) break;
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_2 = PyTuple_GET_ITEM(__pyx_t_1, __pyx_t_7); __Pyx_INCREF(__pyx_t_2); __pyx_t_7++; if (unlikely(0 < 0)) __PYX_ERR(0, 179, __pyx_L1_error)
+ #else
+ __pyx_t_2 = PySequence_ITEM(__pyx_t_1, __pyx_t_7); __pyx_t_7++; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 179, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ #endif
+ }
+ } else {
+ __pyx_t_2 = __pyx_t_9(__pyx_t_1);
+ if (unlikely(!__pyx_t_2)) {
+ PyObject* exc_type = PyErr_Occurred();
+ if (exc_type) {
+ if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear();
+ else __PYX_ERR(0, 179, __pyx_L1_error)
+ }
+ break;
+ }
+ __Pyx_GOTREF(__pyx_t_2);
+ }
+ __Pyx_XDECREF_SET(__pyx_v_i, __pyx_t_2);
+ __pyx_t_2 = 0;
+
+ /* "libmr.pyx":180
+ * assert fit_type in [1, 2, 3, 4]
+ * for i in range(inputDataSize):
+ * svm_data_to_c[i].index = svm_data["labels"][i] # <<<<<<<<<<<<<<
+ * svm_data_to_c[i].value = svm_data["scores"][i]
+ *
+ */
+ __pyx_t_2 = __Pyx_PyObject_Dict_GetItem(__pyx_v_svm_data, __pyx_n_s_labels); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 180, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_4 = __Pyx_PyObject_GetItem(__pyx_t_2, __pyx_v_i); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 180, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __pyx_t_10 = __Pyx_PyInt_As_int(__pyx_t_4); if (unlikely((__pyx_t_10 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 180, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+ __pyx_t_6 = __Pyx_PyIndex_AsSsize_t(__pyx_v_i); if (unlikely((__pyx_t_6 == (Py_ssize_t)-1) && PyErr_Occurred())) __PYX_ERR(0, 180, __pyx_L1_error)
+ (__pyx_v_svm_data_to_c[__pyx_t_6]).index = __pyx_t_10;
+
+ /* "libmr.pyx":181
+ * for i in range(inputDataSize):
+ * svm_data_to_c[i].index = svm_data["labels"][i]
+ * svm_data_to_c[i].value = svm_data["scores"][i] # <<<<<<<<<<<<<<
+ *
+ * print "Data initizalization complete. Now calling C++ code"
+ */
+ __pyx_t_4 = __Pyx_PyObject_Dict_GetItem(__pyx_v_svm_data, __pyx_n_s_scores); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 181, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_2 = __Pyx_PyObject_GetItem(__pyx_t_4, __pyx_v_i); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 181, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+ __pyx_t_11 = __pyx_PyFloat_AsDouble(__pyx_t_2); if (unlikely((__pyx_t_11 == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 181, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __pyx_t_6 = __Pyx_PyIndex_AsSsize_t(__pyx_v_i); if (unlikely((__pyx_t_6 == (Py_ssize_t)-1) && PyErr_Occurred())) __PYX_ERR(0, 181, __pyx_L1_error)
+ (__pyx_v_svm_data_to_c[__pyx_t_6]).value = __pyx_t_11;
+
+ /* "libmr.pyx":179
+ * assert len(svm_data["scores"]) == len(svm_data["labels"])
+ * assert fit_type in [1, 2, 3, 4]
+ * for i in range(inputDataSize): # <<<<<<<<<<<<<<
+ * svm_data_to_c[i].index = svm_data["labels"][i]
+ * svm_data_to_c[i].value = svm_data["scores"][i]
+ */
+ }
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+
+ /* "libmr.pyx":183
+ * svm_data_to_c[i].value = svm_data["scores"][i]
+ *
+ * print "Data initizalization complete. Now calling C++ code" # <<<<<<<<<<<<<<
+ * self.thisptr.FitSVM(svm_data_to_c, inputDataSize, label_of_interest, label_has_positive_score, fit_type, fit_size)
+ * free(svm_data_to_c)
+ */
+ if (__Pyx_PrintOne(0, __pyx_kp_s_Data_initizalization_complete_No) < 0) __PYX_ERR(0, 183, __pyx_L1_error)
+
+ /* "libmr.pyx":184
+ *
+ * print "Data initizalization complete. Now calling C++ code"
+ * self.thisptr.FitSVM(svm_data_to_c, inputDataSize, label_of_interest, label_has_positive_score, fit_type, fit_size) # <<<<<<<<<<<<<<
+ * free(svm_data_to_c)
+ *
+ */
+ __pyx_t_10 = __Pyx_PyInt_As_int(__pyx_v_inputDataSize); if (unlikely((__pyx_t_10 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 184, __pyx_L1_error)
+ __pyx_t_12 = __Pyx_PyInt_As_int(__pyx_v_label_of_interest); if (unlikely((__pyx_t_12 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 184, __pyx_L1_error)
+ __pyx_t_13 = __Pyx_PyObject_IsTrue(__pyx_v_label_has_positive_score); if (unlikely((__pyx_t_13 == ((bool)-1)) && PyErr_Occurred())) __PYX_ERR(0, 184, __pyx_L1_error)
+ __pyx_t_14 = __Pyx_PyInt_As_int(__pyx_v_fit_type); if (unlikely((__pyx_t_14 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 184, __pyx_L1_error)
+ __pyx_t_15 = __Pyx_PyInt_As_int(__pyx_v_fit_size); if (unlikely((__pyx_t_15 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 184, __pyx_L1_error)
+ (void)(__pyx_v_self->thisptr->FitSVM(__pyx_v_svm_data_to_c, __pyx_t_10, __pyx_t_12, __pyx_t_13, __pyx_t_14, __pyx_t_15));
+
+ /* "libmr.pyx":185
+ * print "Data initizalization complete. Now calling C++ code"
+ * self.thisptr.FitSVM(svm_data_to_c, inputDataSize, label_of_interest, label_has_positive_score, fit_type, fit_size)
+ * free(svm_data_to_c) # <<<<<<<<<<<<<<
+ *
+ * property is_valid:
+ */
+ free(__pyx_v_svm_data_to_c);
+
+ /* "libmr.pyx":148
+ * self.thisptr.Load(filetosave)
+ *
+ * def fit_svm(self, svm_data, inputDataSize, label_of_interest, # <<<<<<<<<<<<<<
+ * label_has_positive_score, fit_type, fit_size ):
+ * """
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_AddTraceback("libmr.MR.fit_svm", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_i);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":188
+ *
+ * property is_valid:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.is_valid()
+ * def reset(self):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_8is_valid_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_5libmr_2MR_8is_valid_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_8is_valid___get__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_8is_valid___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "libmr.pyx":189
+ * property is_valid:
+ * def __get__(self):
+ * return self.thisptr.is_valid() # <<<<<<<<<<<<<<
+ * def reset(self):
+ * self.thisptr.Reset()
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyBool_FromLong(__pyx_v_self->thisptr->is_valid()); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 189, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":188
+ *
+ * property is_valid:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.is_valid()
+ * def reset(self):
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.is_valid.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":190
+ * def __get__(self):
+ * return self.thisptr.is_valid()
+ * def reset(self): # <<<<<<<<<<<<<<
+ * self.thisptr.Reset()
+ * def predict_match(self, double x, double threshold = .9999999):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_15reset(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/
+static PyObject *__pyx_pw_5libmr_2MR_15reset(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("reset (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_14reset(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_14reset(struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("reset", 0);
+
+ /* "libmr.pyx":191
+ * return self.thisptr.is_valid()
+ * def reset(self):
+ * self.thisptr.Reset() # <<<<<<<<<<<<<<
+ * def predict_match(self, double x, double threshold = .9999999):
+ * """
+ */
+ __pyx_v_self->thisptr->Reset();
+
+ /* "libmr.pyx":190
+ * def __get__(self):
+ * return self.thisptr.is_valid()
+ * def reset(self): # <<<<<<<<<<<<<<
+ * self.thisptr.Reset()
+ * def predict_match(self, double x, double threshold = .9999999):
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":192
+ * def reset(self):
+ * self.thisptr.Reset()
+ * def predict_match(self, double x, double threshold = .9999999): # <<<<<<<<<<<<<<
+ * """
+ * Is X from the "match" distribution (i.e. we reject null hypothesis
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_17predict_match(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
+static char __pyx_doc_5libmr_2MR_16predict_match[] = "\n Is X from the \"match\" distribution (i.e. we reject null hypothesis\n of non-match)\n\n ";
+static PyObject *__pyx_pw_5libmr_2MR_17predict_match(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
+ double __pyx_v_x;
+ double __pyx_v_threshold;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("predict_match (wrapper)", 0);
+ {
+ static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_x,&__pyx_n_s_threshold,0};
+ PyObject* values[2] = {0,0};
+ if (unlikely(__pyx_kwds)) {
+ Py_ssize_t kw_args;
+ const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
+ switch (pos_args) {
+ case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ CYTHON_FALLTHROUGH;
+ case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ CYTHON_FALLTHROUGH;
+ case 0: break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ kw_args = PyDict_Size(__pyx_kwds);
+ switch (pos_args) {
+ case 0:
+ if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_x)) != 0)) kw_args--;
+ else goto __pyx_L5_argtuple_error;
+ CYTHON_FALLTHROUGH;
+ case 1:
+ if (kw_args > 0) {
+ PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_threshold);
+ if (value) { values[1] = value; kw_args--; }
+ }
+ }
+ if (unlikely(kw_args > 0)) {
+ if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "predict_match") < 0)) __PYX_ERR(0, 192, __pyx_L3_error)
+ }
+ } else {
+ switch (PyTuple_GET_SIZE(__pyx_args)) {
+ case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ CYTHON_FALLTHROUGH;
+ case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ }
+ __pyx_v_x = __pyx_PyFloat_AsDouble(values[0]); if (unlikely((__pyx_v_x == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 192, __pyx_L3_error)
+ if (values[1]) {
+ __pyx_v_threshold = __pyx_PyFloat_AsDouble(values[1]); if (unlikely((__pyx_v_threshold == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 192, __pyx_L3_error)
+ } else {
+ __pyx_v_threshold = ((double).9999999);
+ }
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L5_argtuple_error:;
+ __Pyx_RaiseArgtupleInvalid("predict_match", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 192, __pyx_L3_error)
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.predict_match", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return NULL;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_16predict_match(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), __pyx_v_x, __pyx_v_threshold);
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_16predict_match(struct __pyx_obj_5libmr_MR *__pyx_v_self, double __pyx_v_x, double __pyx_v_threshold) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("predict_match", 0);
+
+ /* "libmr.pyx":198
+ *
+ * """
+ * return self.thisptr.Predict_Match(x,threshold) # <<<<<<<<<<<<<<
+ * def w_score(self, double x):
+ * """
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyBool_FromLong(__pyx_v_self->thisptr->Predict_Match(__pyx_v_x, __pyx_v_threshold)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 198, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":192
+ * def reset(self):
+ * self.thisptr.Reset()
+ * def predict_match(self, double x, double threshold = .9999999): # <<<<<<<<<<<<<<
+ * """
+ * Is X from the "match" distribution (i.e. we reject null hypothesis
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.predict_match", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":199
+ * """
+ * return self.thisptr.Predict_Match(x,threshold)
+ * def w_score(self, double x): # <<<<<<<<<<<<<<
+ * """
+ * This is the commonly used function. After fitting, it returns the probability of the given score being "correct". It is the same as CDF
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_19w_score(PyObject *__pyx_v_self, PyObject *__pyx_arg_x); /*proto*/
+static char __pyx_doc_5libmr_2MR_18w_score[] = "\n\tThis is the commonly used function. After fitting, it returns the probability of the given score being \"correct\". It is the same as CDF\n ";
+static PyObject *__pyx_pw_5libmr_2MR_19w_score(PyObject *__pyx_v_self, PyObject *__pyx_arg_x) {
+ double __pyx_v_x;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("w_score (wrapper)", 0);
+ assert(__pyx_arg_x); {
+ __pyx_v_x = __pyx_PyFloat_AsDouble(__pyx_arg_x); if (unlikely((__pyx_v_x == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 199, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.w_score", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return NULL;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_18w_score(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((double)__pyx_v_x));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_18w_score(struct __pyx_obj_5libmr_MR *__pyx_v_self, double __pyx_v_x) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("w_score", 0);
+
+ /* "libmr.pyx":203
+ * This is the commonly used function. After fitting, it returns the probability of the given score being "correct". It is the same as CDF
+ * """
+ * return self.thisptr.W_score(x) # <<<<<<<<<<<<<<
+ * def cdf(self, double x):
+ * """
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyFloat_FromDouble(__pyx_v_self->thisptr->W_score(__pyx_v_x)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 203, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":199
+ * """
+ * return self.thisptr.Predict_Match(x,threshold)
+ * def w_score(self, double x): # <<<<<<<<<<<<<<
+ * """
+ * This is the commonly used function. After fitting, it returns the probability of the given score being "correct". It is the same as CDF
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.w_score", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":204
+ * """
+ * return self.thisptr.W_score(x)
+ * def cdf(self, double x): # <<<<<<<<<<<<<<
+ * """
+ * This is the cummumlative probablity of match being corrrect (or more precisely the probility the score (after transform) being an outlier for the distribution, which given the transforms applied, so bigger is better, this is the probablity the score is correct.
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_21cdf(PyObject *__pyx_v_self, PyObject *__pyx_arg_x); /*proto*/
+static char __pyx_doc_5libmr_2MR_20cdf[] = "\n This is the cummumlative probablity of match being corrrect (or more precisely the probility the score (after transform) being an outlier for the distribution, which given the transforms applied, so bigger is better, this is the probablity the score is correct.\n ";
+static PyObject *__pyx_pw_5libmr_2MR_21cdf(PyObject *__pyx_v_self, PyObject *__pyx_arg_x) {
+ double __pyx_v_x;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("cdf (wrapper)", 0);
+ assert(__pyx_arg_x); {
+ __pyx_v_x = __pyx_PyFloat_AsDouble(__pyx_arg_x); if (unlikely((__pyx_v_x == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 204, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.cdf", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return NULL;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_20cdf(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((double)__pyx_v_x));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_20cdf(struct __pyx_obj_5libmr_MR *__pyx_v_self, double __pyx_v_x) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("cdf", 0);
+
+ /* "libmr.pyx":208
+ * This is the cummumlative probablity of match being corrrect (or more precisely the probility the score (after transform) being an outlier for the distribution, which given the transforms applied, so bigger is better, this is the probablity the score is correct.
+ * """
+ * return self.thisptr.CDF(x) # <<<<<<<<<<<<<<
+ * def inv(self, double p):
+ * """
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyFloat_FromDouble(__pyx_v_self->thisptr->CDF(__pyx_v_x)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 208, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":204
+ * """
+ * return self.thisptr.W_score(x)
+ * def cdf(self, double x): # <<<<<<<<<<<<<<
+ * """
+ * This is the cummumlative probablity of match being corrrect (or more precisely the probility the score (after transform) being an outlier for the distribution, which given the transforms applied, so bigger is better, this is the probablity the score is correct.
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.cdf", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":209
+ * """
+ * return self.thisptr.CDF(x)
+ * def inv(self, double p): # <<<<<<<<<<<<<<
+ * """
+ * This is score for which one would obtain CDF probability p (i.e. x such that p = CDF(x))
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_23inv(PyObject *__pyx_v_self, PyObject *__pyx_arg_p); /*proto*/
+static char __pyx_doc_5libmr_2MR_22inv[] = "\n This is score for which one would obtain CDF probability p (i.e. x such that p = CDF(x))\n ";
+static PyObject *__pyx_pw_5libmr_2MR_23inv(PyObject *__pyx_v_self, PyObject *__pyx_arg_p) {
+ double __pyx_v_p;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("inv (wrapper)", 0);
+ assert(__pyx_arg_p); {
+ __pyx_v_p = __pyx_PyFloat_AsDouble(__pyx_arg_p); if (unlikely((__pyx_v_p == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 209, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.inv", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return NULL;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_22inv(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((double)__pyx_v_p));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_22inv(struct __pyx_obj_5libmr_MR *__pyx_v_self, double __pyx_v_p) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("inv", 0);
+
+ /* "libmr.pyx":213
+ * This is score for which one would obtain CDF probability p (i.e. x such that p = CDF(x))
+ * """
+ * return self.thisptr.Inv(p) # <<<<<<<<<<<<<<
+ * def w_score_vector(self, double[::1] invec):
+ * """
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyFloat_FromDouble(__pyx_v_self->thisptr->Inv(__pyx_v_p)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 213, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":209
+ * """
+ * return self.thisptr.CDF(x)
+ * def inv(self, double p): # <<<<<<<<<<<<<<
+ * """
+ * This is score for which one would obtain CDF probability p (i.e. x such that p = CDF(x))
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.inv", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":214
+ * """
+ * return self.thisptr.Inv(p)
+ * def w_score_vector(self, double[::1] invec): # <<<<<<<<<<<<<<
+ * """
+ * Apply w_score to each element of invec, returning a new vector of W-scores
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_25w_score_vector(PyObject *__pyx_v_self, PyObject *__pyx_arg_invec); /*proto*/
+static char __pyx_doc_5libmr_2MR_24w_score_vector[] = "\n Apply w_score to each element of invec, returning a new vector of W-scores\n ";
+static PyObject *__pyx_pw_5libmr_2MR_25w_score_vector(PyObject *__pyx_v_self, PyObject *__pyx_arg_invec) {
+ __Pyx_memviewslice __pyx_v_invec = { 0, 0, { 0 }, { 0 }, { 0 } };
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("w_score_vector (wrapper)", 0);
+ assert(__pyx_arg_invec); {
+ __pyx_v_invec = __Pyx_PyObject_to_MemoryviewSlice_dc_double(__pyx_arg_invec, PyBUF_WRITABLE); if (unlikely(!__pyx_v_invec.memview)) __PYX_ERR(0, 214, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.w_score_vector", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return NULL;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_24w_score_vector(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), __pyx_v_invec);
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_24w_score_vector(struct __pyx_obj_5libmr_MR *__pyx_v_self, __Pyx_memviewslice __pyx_v_invec) {
+ PyArrayObject *__pyx_v_new_vec = 0;
+ __Pyx_LocalBuf_ND __pyx_pybuffernd_new_vec;
+ __Pyx_Buffer __pyx_pybuffer_new_vec;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ size_t __pyx_t_3;
+ PyObject *__pyx_t_4 = NULL;
+ PyObject *__pyx_t_5 = NULL;
+ PyArrayObject *__pyx_t_6 = NULL;
+ Py_ssize_t __pyx_t_7;
+ int __pyx_t_8;
+ Py_ssize_t __pyx_t_9;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("w_score_vector", 0);
+ __pyx_pybuffer_new_vec.pybuffer.buf = NULL;
+ __pyx_pybuffer_new_vec.refcount = 0;
+ __pyx_pybuffernd_new_vec.data = NULL;
+ __pyx_pybuffernd_new_vec.rcbuffer = &__pyx_pybuffer_new_vec;
+
+ /* "libmr.pyx":218
+ * Apply w_score to each element of invec, returning a new vector of W-scores
+ * """
+ * cdef np.ndarray[np.double_t,ndim=1]new_vec = np.zeros(len(invec), dtype='d') # <<<<<<<<<<<<<<
+ * self.thisptr.ReNormalize(&invec[0], &new_vec[0], len(invec))
+ * return new_vec
+ */
+ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_np); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 218, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_zeros); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 218, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_t_3 = __Pyx_MemoryView_Len(__pyx_v_invec);
+ __pyx_t_1 = __Pyx_PyInt_FromSize_t(__pyx_t_3); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 218, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_4 = PyTuple_New(1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 218, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_GIVEREF(__pyx_t_1);
+ PyTuple_SET_ITEM(__pyx_t_4, 0, __pyx_t_1);
+ __pyx_t_1 = 0;
+ __pyx_t_1 = __Pyx_PyDict_NewPresized(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 218, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ if (PyDict_SetItem(__pyx_t_1, __pyx_n_s_dtype, __pyx_n_s_d) < 0) __PYX_ERR(0, 218, __pyx_L1_error)
+ __pyx_t_5 = __Pyx_PyObject_Call(__pyx_t_2, __pyx_t_4, __pyx_t_1); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 218, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ if (!(likely(((__pyx_t_5) == Py_None) || likely(__Pyx_TypeTest(__pyx_t_5, __pyx_ptype_5numpy_ndarray))))) __PYX_ERR(0, 218, __pyx_L1_error)
+ __pyx_t_6 = ((PyArrayObject *)__pyx_t_5);
+ {
+ __Pyx_BufFmt_StackElem __pyx_stack[1];
+ if (unlikely(__Pyx_GetBufferAndValidate(&__pyx_pybuffernd_new_vec.rcbuffer->pybuffer, (PyObject*)__pyx_t_6, &__Pyx_TypeInfo_nn___pyx_t_5numpy_double_t, PyBUF_FORMAT| PyBUF_STRIDES, 1, 0, __pyx_stack) == -1)) {
+ __pyx_v_new_vec = ((PyArrayObject *)Py_None); __Pyx_INCREF(Py_None); __pyx_pybuffernd_new_vec.rcbuffer->pybuffer.buf = NULL;
+ __PYX_ERR(0, 218, __pyx_L1_error)
+ } else {__pyx_pybuffernd_new_vec.diminfo[0].strides = __pyx_pybuffernd_new_vec.rcbuffer->pybuffer.strides[0]; __pyx_pybuffernd_new_vec.diminfo[0].shape = __pyx_pybuffernd_new_vec.rcbuffer->pybuffer.shape[0];
+ }
+ }
+ __pyx_t_6 = 0;
+ __pyx_v_new_vec = ((PyArrayObject *)__pyx_t_5);
+ __pyx_t_5 = 0;
+
+ /* "libmr.pyx":219
+ * """
+ * cdef np.ndarray[np.double_t,ndim=1]new_vec = np.zeros(len(invec), dtype='d')
+ * self.thisptr.ReNormalize(&invec[0], &new_vec[0], len(invec)) # <<<<<<<<<<<<<<
+ * return new_vec
+ * def __str__(self):
+ */
+ __pyx_t_7 = 0;
+ __pyx_t_8 = -1;
+ if (__pyx_t_7 < 0) {
+ __pyx_t_7 += __pyx_v_invec.shape[0];
+ if (unlikely(__pyx_t_7 < 0)) __pyx_t_8 = 0;
+ } else if (unlikely(__pyx_t_7 >= __pyx_v_invec.shape[0])) __pyx_t_8 = 0;
+ if (unlikely(__pyx_t_8 != -1)) {
+ __Pyx_RaiseBufferIndexError(__pyx_t_8);
+ __PYX_ERR(0, 219, __pyx_L1_error)
+ }
+ __pyx_t_9 = 0;
+ __pyx_t_8 = -1;
+ if (__pyx_t_9 < 0) {
+ __pyx_t_9 += __pyx_pybuffernd_new_vec.diminfo[0].shape;
+ if (unlikely(__pyx_t_9 < 0)) __pyx_t_8 = 0;
+ } else if (unlikely(__pyx_t_9 >= __pyx_pybuffernd_new_vec.diminfo[0].shape)) __pyx_t_8 = 0;
+ if (unlikely(__pyx_t_8 != -1)) {
+ __Pyx_RaiseBufferIndexError(__pyx_t_8);
+ __PYX_ERR(0, 219, __pyx_L1_error)
+ }
+ __pyx_t_3 = __Pyx_MemoryView_Len(__pyx_v_invec);
+ (void)(__pyx_v_self->thisptr->ReNormalize((&(*((double *) ( /* dim=0 */ ((char *) (((double *) __pyx_v_invec.data) + __pyx_t_7)) )))), (&(*__Pyx_BufPtrStrided1d(__pyx_t_5numpy_double_t *, __pyx_pybuffernd_new_vec.rcbuffer->pybuffer.buf, __pyx_t_9, __pyx_pybuffernd_new_vec.diminfo[0].strides))), __pyx_t_3));
+
+ /* "libmr.pyx":220
+ * cdef np.ndarray[np.double_t,ndim=1]new_vec = np.zeros(len(invec), dtype='d')
+ * self.thisptr.ReNormalize(&invec[0], &new_vec[0], len(invec))
+ * return new_vec # <<<<<<<<<<<<<<
+ * def __str__(self):
+ * """
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(((PyObject *)__pyx_v_new_vec));
+ __pyx_r = ((PyObject *)__pyx_v_new_vec);
+ goto __pyx_L0;
+
+ /* "libmr.pyx":214
+ * """
+ * return self.thisptr.Inv(p)
+ * def w_score_vector(self, double[::1] invec): # <<<<<<<<<<<<<<
+ * """
+ * Apply w_score to each element of invec, returning a new vector of W-scores
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_XDECREF(__pyx_t_5);
+ { PyObject *__pyx_type, *__pyx_value, *__pyx_tb;
+ __Pyx_PyThreadState_declare
+ __Pyx_PyThreadState_assign
+ __Pyx_ErrFetch(&__pyx_type, &__pyx_value, &__pyx_tb);
+ __Pyx_SafeReleaseBuffer(&__pyx_pybuffernd_new_vec.rcbuffer->pybuffer);
+ __Pyx_ErrRestore(__pyx_type, __pyx_value, __pyx_tb);}
+ __Pyx_AddTraceback("libmr.MR.w_score_vector", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ goto __pyx_L2;
+ __pyx_L0:;
+ __Pyx_SafeReleaseBuffer(&__pyx_pybuffernd_new_vec.rcbuffer->pybuffer);
+ __pyx_L2:;
+ __PYX_XDEC_MEMVIEW(&__pyx_v_invec, 1);
+ __Pyx_XDECREF((PyObject *)__pyx_v_new_vec);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":221
+ * self.thisptr.ReNormalize(&invec[0], &new_vec[0], len(invec))
+ * return new_vec
+ * def __str__(self): # <<<<<<<<<<<<<<
+ * """
+ * Serialize the MR object to a string. Use load_from_string to recover it.
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_27__str__(PyObject *__pyx_v_self); /*proto*/
+static char __pyx_doc_5libmr_2MR_26__str__[] = "\n Serialize the MR object to a string. Use load_from_string to recover it.\n ";
+#if CYTHON_COMPILING_IN_CPYTHON
+struct wrapperbase __pyx_wrapperbase_5libmr_2MR_26__str__;
+#endif
+static PyObject *__pyx_pw_5libmr_2MR_27__str__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__str__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_26__str__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_26__str__(struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__str__", 0);
+
+ /* "libmr.pyx":225
+ * Serialize the MR object to a string. Use load_from_string to recover it.
+ * """
+ * return self.thisptr.to_string() # <<<<<<<<<<<<<<
+ * def __repr__(self):
+ * return "" % str(self)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __pyx_convert_PyBytes_string_to_py_std__in_string(__pyx_v_self->thisptr->to_string()); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 225, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":221
+ * self.thisptr.ReNormalize(&invec[0], &new_vec[0], len(invec))
+ * return new_vec
+ * def __str__(self): # <<<<<<<<<<<<<<
+ * """
+ * Serialize the MR object to a string. Use load_from_string to recover it.
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.__str__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":226
+ * """
+ * return self.thisptr.to_string()
+ * def __repr__(self): # <<<<<<<<<<<<<<
+ * return "" % str(self)
+ * property tailsize:
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_29__repr__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_5libmr_2MR_29__repr__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__repr__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_28__repr__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_28__repr__(struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__repr__", 0);
+
+ /* "libmr.pyx":227
+ * return self.thisptr.to_string()
+ * def __repr__(self):
+ * return "" % str(self) # <<<<<<<<<<<<<<
+ * property tailsize:
+ * def __get__(self):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyString_Type)), ((PyObject *)__pyx_v_self)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 227, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_2 = __Pyx_PyString_FormatSafe(__pyx_kp_s_MR_object_r, __pyx_t_1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 227, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":226
+ * """
+ * return self.thisptr.to_string()
+ * def __repr__(self): # <<<<<<<<<<<<<<
+ * return "" % str(self)
+ * property tailsize:
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_AddTraceback("libmr.MR.__repr__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":229
+ * return "" % str(self)
+ * property tailsize:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.get_fitting_size()
+ * def __set__(self, int nsize):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_8tailsize_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_5libmr_2MR_8tailsize_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_8tailsize___get__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_8tailsize___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "libmr.pyx":230
+ * property tailsize:
+ * def __get__(self):
+ * return self.thisptr.get_fitting_size() # <<<<<<<<<<<<<<
+ * def __set__(self, int nsize):
+ * self.thisptr.set_fitting_size(nsize)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->thisptr->get_fitting_size()); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 230, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":229
+ * return "" % str(self)
+ * property tailsize:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.get_fitting_size()
+ * def __set__(self, int nsize):
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.tailsize.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":231
+ * def __get__(self):
+ * return self.thisptr.get_fitting_size()
+ * def __set__(self, int nsize): # <<<<<<<<<<<<<<
+ * self.thisptr.set_fitting_size(nsize)
+ * property translate_amount:
+ */
+
+/* Python wrapper */
+static int __pyx_pw_5libmr_2MR_8tailsize_3__set__(PyObject *__pyx_v_self, PyObject *__pyx_arg_nsize); /*proto*/
+static int __pyx_pw_5libmr_2MR_8tailsize_3__set__(PyObject *__pyx_v_self, PyObject *__pyx_arg_nsize) {
+ int __pyx_v_nsize;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__set__ (wrapper)", 0);
+ assert(__pyx_arg_nsize); {
+ __pyx_v_nsize = __Pyx_PyInt_As_int(__pyx_arg_nsize); if (unlikely((__pyx_v_nsize == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 231, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.tailsize.__set__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return -1;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_8tailsize_2__set__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((int)__pyx_v_nsize));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_pf_5libmr_2MR_8tailsize_2__set__(struct __pyx_obj_5libmr_MR *__pyx_v_self, int __pyx_v_nsize) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__set__", 0);
+
+ /* "libmr.pyx":232
+ * return self.thisptr.get_fitting_size()
+ * def __set__(self, int nsize):
+ * self.thisptr.set_fitting_size(nsize) # <<<<<<<<<<<<<<
+ * property translate_amount:
+ * def __get__(self):
+ */
+ (void)(__pyx_v_self->thisptr->set_fitting_size(__pyx_v_nsize));
+
+ /* "libmr.pyx":231
+ * def __get__(self):
+ * return self.thisptr.get_fitting_size()
+ * def __set__(self, int nsize): # <<<<<<<<<<<<<<
+ * self.thisptr.set_fitting_size(nsize)
+ * property translate_amount:
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":234
+ * self.thisptr.set_fitting_size(nsize)
+ * property translate_amount:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.get_translate_amount()
+ * def __set__(self, int ntrans):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_16translate_amount_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_5libmr_2MR_16translate_amount_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_16translate_amount___get__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_16translate_amount___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "libmr.pyx":235
+ * property translate_amount:
+ * def __get__(self):
+ * return self.thisptr.get_translate_amount() # <<<<<<<<<<<<<<
+ * def __set__(self, int ntrans):
+ * self.thisptr.set_translate_amount(ntrans)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->thisptr->get_translate_amount()); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 235, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":234
+ * self.thisptr.set_fitting_size(nsize)
+ * property translate_amount:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.get_translate_amount()
+ * def __set__(self, int ntrans):
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.translate_amount.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":236
+ * def __get__(self):
+ * return self.thisptr.get_translate_amount()
+ * def __set__(self, int ntrans): # <<<<<<<<<<<<<<
+ * self.thisptr.set_translate_amount(ntrans)
+ * property sign:
+ */
+
+/* Python wrapper */
+static int __pyx_pw_5libmr_2MR_16translate_amount_3__set__(PyObject *__pyx_v_self, PyObject *__pyx_arg_ntrans); /*proto*/
+static int __pyx_pw_5libmr_2MR_16translate_amount_3__set__(PyObject *__pyx_v_self, PyObject *__pyx_arg_ntrans) {
+ int __pyx_v_ntrans;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__set__ (wrapper)", 0);
+ assert(__pyx_arg_ntrans); {
+ __pyx_v_ntrans = __Pyx_PyInt_As_int(__pyx_arg_ntrans); if (unlikely((__pyx_v_ntrans == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 236, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.translate_amount.__set__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return -1;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_16translate_amount_2__set__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((int)__pyx_v_ntrans));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_pf_5libmr_2MR_16translate_amount_2__set__(struct __pyx_obj_5libmr_MR *__pyx_v_self, int __pyx_v_ntrans) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__set__", 0);
+
+ /* "libmr.pyx":237
+ * return self.thisptr.get_translate_amount()
+ * def __set__(self, int ntrans):
+ * self.thisptr.set_translate_amount(ntrans) # <<<<<<<<<<<<<<
+ * property sign:
+ * def __get__(self):
+ */
+ (void)(__pyx_v_self->thisptr->set_translate_amount(__pyx_v_ntrans));
+
+ /* "libmr.pyx":236
+ * def __get__(self):
+ * return self.thisptr.get_translate_amount()
+ * def __set__(self, int ntrans): # <<<<<<<<<<<<<<
+ * self.thisptr.set_translate_amount(ntrans)
+ * property sign:
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":239
+ * self.thisptr.set_translate_amount(ntrans)
+ * property sign:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.get_sign()
+ * def __set__(self, int nsign):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_4sign_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_5libmr_2MR_4sign_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_4sign___get__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_4sign___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "libmr.pyx":240
+ * property sign:
+ * def __get__(self):
+ * return self.thisptr.get_sign() # <<<<<<<<<<<<<<
+ * def __set__(self, int nsign):
+ * self.thisptr.set_sign(nsign)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->thisptr->get_sign()); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 240, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":239
+ * self.thisptr.set_translate_amount(ntrans)
+ * property sign:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.get_sign()
+ * def __set__(self, int nsign):
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.sign.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":241
+ * def __get__(self):
+ * return self.thisptr.get_sign()
+ * def __set__(self, int nsign): # <<<<<<<<<<<<<<
+ * self.thisptr.set_sign(nsign)
+ * property small_score:
+ */
+
+/* Python wrapper */
+static int __pyx_pw_5libmr_2MR_4sign_3__set__(PyObject *__pyx_v_self, PyObject *__pyx_arg_nsign); /*proto*/
+static int __pyx_pw_5libmr_2MR_4sign_3__set__(PyObject *__pyx_v_self, PyObject *__pyx_arg_nsign) {
+ int __pyx_v_nsign;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__set__ (wrapper)", 0);
+ assert(__pyx_arg_nsign); {
+ __pyx_v_nsign = __Pyx_PyInt_As_int(__pyx_arg_nsign); if (unlikely((__pyx_v_nsign == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 241, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.sign.__set__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return -1;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_4sign_2__set__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((int)__pyx_v_nsign));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_pf_5libmr_2MR_4sign_2__set__(struct __pyx_obj_5libmr_MR *__pyx_v_self, int __pyx_v_nsign) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__set__", 0);
+
+ /* "libmr.pyx":242
+ * return self.thisptr.get_sign()
+ * def __set__(self, int nsign):
+ * self.thisptr.set_sign(nsign) # <<<<<<<<<<<<<<
+ * property small_score:
+ * def __get__(self):
+ */
+ (void)(__pyx_v_self->thisptr->set_sign(__pyx_v_nsign));
+
+ /* "libmr.pyx":241
+ * def __get__(self):
+ * return self.thisptr.get_sign()
+ * def __set__(self, int nsign): # <<<<<<<<<<<<<<
+ * self.thisptr.set_sign(nsign)
+ * property small_score:
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":244
+ * self.thisptr.set_sign(nsign)
+ * property small_score:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.get_small_score()
+ * def __set__(self, double nscore):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_11small_score_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_5libmr_2MR_11small_score_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_11small_score___get__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_11small_score___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "libmr.pyx":245
+ * property small_score:
+ * def __get__(self):
+ * return self.thisptr.get_small_score() # <<<<<<<<<<<<<<
+ * def __set__(self, double nscore):
+ * self.thisptr.set_small_score(nscore)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyFloat_FromDouble(__pyx_v_self->thisptr->get_small_score()); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 245, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":244
+ * self.thisptr.set_sign(nsign)
+ * property small_score:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.get_small_score()
+ * def __set__(self, double nscore):
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.small_score.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":246
+ * def __get__(self):
+ * return self.thisptr.get_small_score()
+ * def __set__(self, double nscore): # <<<<<<<<<<<<<<
+ * self.thisptr.set_small_score(nscore)
+ * property verbose:
+ */
+
+/* Python wrapper */
+static int __pyx_pw_5libmr_2MR_11small_score_3__set__(PyObject *__pyx_v_self, PyObject *__pyx_arg_nscore); /*proto*/
+static int __pyx_pw_5libmr_2MR_11small_score_3__set__(PyObject *__pyx_v_self, PyObject *__pyx_arg_nscore) {
+ double __pyx_v_nscore;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__set__ (wrapper)", 0);
+ assert(__pyx_arg_nscore); {
+ __pyx_v_nscore = __pyx_PyFloat_AsDouble(__pyx_arg_nscore); if (unlikely((__pyx_v_nscore == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 246, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.small_score.__set__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return -1;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_11small_score_2__set__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((double)__pyx_v_nscore));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_pf_5libmr_2MR_11small_score_2__set__(struct __pyx_obj_5libmr_MR *__pyx_v_self, double __pyx_v_nscore) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__set__", 0);
+
+ /* "libmr.pyx":247
+ * return self.thisptr.get_small_score()
+ * def __set__(self, double nscore):
+ * self.thisptr.set_small_score(nscore) # <<<<<<<<<<<<<<
+ * property verbose:
+ * def __get__(self):
+ */
+ (void)(__pyx_v_self->thisptr->set_small_score(__pyx_v_nscore));
+
+ /* "libmr.pyx":246
+ * def __get__(self):
+ * return self.thisptr.get_small_score()
+ * def __set__(self, double nscore): # <<<<<<<<<<<<<<
+ * self.thisptr.set_small_score(nscore)
+ * property verbose:
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":249
+ * self.thisptr.set_small_score(nscore)
+ * property verbose:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.verbose
+ * def __set__(self, bool verbose):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_7verbose_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_5libmr_2MR_7verbose_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_7verbose___get__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_7verbose___get__(struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "libmr.pyx":250
+ * property verbose:
+ * def __get__(self):
+ * return self.thisptr.verbose # <<<<<<<<<<<<<<
+ * def __set__(self, bool verbose):
+ * self.thisptr.verbose = verbose
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyBool_FromLong(__pyx_v_self->thisptr->verbose); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 250, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "libmr.pyx":249
+ * self.thisptr.set_small_score(nscore)
+ * property verbose:
+ * def __get__(self): # <<<<<<<<<<<<<<
+ * return self.thisptr.verbose
+ * def __set__(self, bool verbose):
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.verbose.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":251
+ * def __get__(self):
+ * return self.thisptr.verbose
+ * def __set__(self, bool verbose): # <<<<<<<<<<<<<<
+ * self.thisptr.verbose = verbose
+ *
+ */
+
+/* Python wrapper */
+static int __pyx_pw_5libmr_2MR_7verbose_3__set__(PyObject *__pyx_v_self, PyObject *__pyx_arg_verbose); /*proto*/
+static int __pyx_pw_5libmr_2MR_7verbose_3__set__(PyObject *__pyx_v_self, PyObject *__pyx_arg_verbose) {
+ bool __pyx_v_verbose;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__set__ (wrapper)", 0);
+ assert(__pyx_arg_verbose); {
+ __pyx_v_verbose = __Pyx_PyObject_IsTrue(__pyx_arg_verbose); if (unlikely((__pyx_v_verbose == ((bool)-1)) && PyErr_Occurred())) __PYX_ERR(0, 251, __pyx_L3_error)
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("libmr.MR.verbose.__set__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return -1;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_pf_5libmr_2MR_7verbose_2__set__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((bool)__pyx_v_verbose));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_pf_5libmr_2MR_7verbose_2__set__(struct __pyx_obj_5libmr_MR *__pyx_v_self, bool __pyx_v_verbose) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__set__", 0);
+
+ /* "libmr.pyx":252
+ * return self.thisptr.verbose
+ * def __set__(self, bool verbose):
+ * self.thisptr.verbose = verbose # <<<<<<<<<<<<<<
+ *
+ * def load_from_string(str input):
+ */
+ __pyx_v_self->thisptr->verbose = __pyx_v_verbose;
+
+ /* "libmr.pyx":251
+ * def __get__(self):
+ * return self.thisptr.verbose
+ * def __set__(self, bool verbose): # <<<<<<<<<<<<<<
+ * self.thisptr.verbose = verbose
+ *
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "(tree fragment)":1
+ * def __reduce_cython__(self): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_31__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/
+static PyObject *__pyx_pw_5libmr_2MR_31__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_30__reduce_cython__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_30__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_5libmr_MR *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__reduce_cython__", 0);
+
+ /* "(tree fragment)":2
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<<
+ * def __setstate_cython__(self, __pyx_state):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple_, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 2, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_Raise(__pyx_t_1, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __PYX_ERR(1, 2, __pyx_L1_error)
+
+ /* "(tree fragment)":1
+ * def __reduce_cython__(self): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "(tree fragment)":3
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_2MR_33__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/
+static PyObject *__pyx_pw_5libmr_2MR_33__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_5libmr_2MR_32__setstate_cython__(((struct __pyx_obj_5libmr_MR *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_2MR_32__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_5libmr_MR *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__setstate_cython__", 0);
+
+ /* "(tree fragment)":4
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<<
+ */
+ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__2, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_Raise(__pyx_t_1, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __PYX_ERR(1, 4, __pyx_L1_error)
+
+ /* "(tree fragment)":3
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.MR.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "libmr.pyx":254
+ * self.thisptr.verbose = verbose
+ *
+ * def load_from_string(str input): # <<<<<<<<<<<<<<
+ * """
+ * Deserialize an MR object. This turns a string back into an MR object; it is the inverse of str(MR())
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_5libmr_1load_from_string(PyObject *__pyx_self, PyObject *__pyx_v_input); /*proto*/
+static char __pyx_doc_5libmr_load_from_string[] = "\n Deserialize an MR object. This turns a string back into an MR object; it is the inverse of str(MR())\n ";
+static PyMethodDef __pyx_mdef_5libmr_1load_from_string = {"load_from_string", (PyCFunction)__pyx_pw_5libmr_1load_from_string, METH_O, __pyx_doc_5libmr_load_from_string};
+static PyObject *__pyx_pw_5libmr_1load_from_string(PyObject *__pyx_self, PyObject *__pyx_v_input) {
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("load_from_string (wrapper)", 0);
+ if (unlikely(!__Pyx_ArgTypeTest(((PyObject *)__pyx_v_input), (&PyString_Type), 1, "input", 1))) __PYX_ERR(0, 254, __pyx_L1_error)
+ __pyx_r = __pyx_pf_5libmr_load_from_string(__pyx_self, ((PyObject*)__pyx_v_input));
+
+ /* function exit code */
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_5libmr_load_from_string(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_input) {
+ struct __pyx_obj_5libmr_MR *__pyx_v_pymr = NULL;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ std::string __pyx_t_2;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("load_from_string", 0);
+
+ /* "libmr.pyx":258
+ * Deserialize an MR object. This turns a string back into an MR object; it is the inverse of str(MR())
+ * """
+ * pymr = MR() # <<<<<<<<<<<<<<
+ * pymr.thisptr.from_string(input)
+ * return pymr
+ */
+ __pyx_t_1 = __Pyx_PyObject_CallNoArg(((PyObject *)__pyx_ptype_5libmr_MR)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 258, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_v_pymr = ((struct __pyx_obj_5libmr_MR *)__pyx_t_1);
+ __pyx_t_1 = 0;
+
+ /* "libmr.pyx":259
+ * """
+ * pymr = MR()
+ * pymr.thisptr.from_string(input) # <<<<<<<<<<<<<<
+ * return pymr
+ *
+ */
+ __pyx_t_2 = __pyx_convert_string_from_py_std__in_string(__pyx_v_input); if (unlikely(PyErr_Occurred())) __PYX_ERR(0, 259, __pyx_L1_error)
+ __pyx_v_pymr->thisptr->from_string(__pyx_t_2);
+
+ /* "libmr.pyx":260
+ * pymr = MR()
+ * pymr.thisptr.from_string(input)
+ * return pymr # <<<<<<<<<<<<<<
+ *
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(((PyObject *)__pyx_v_pymr));
+ __pyx_r = ((PyObject *)__pyx_v_pymr);
+ goto __pyx_L0;
+
+ /* "libmr.pyx":254
+ * self.thisptr.verbose = verbose
+ *
+ * def load_from_string(str input): # <<<<<<<<<<<<<<
+ * """
+ * Deserialize an MR object. This turns a string back into an MR object; it is the inverse of str(MR())
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("libmr.load_from_string", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XDECREF((PyObject *)__pyx_v_pymr);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":734
+ * ctypedef npy_cdouble complex_t
+ *
+ * cdef inline object PyArray_MultiIterNew1(a): # <<<<<<<<<<<<<<
+ * return PyArray_MultiIterNew(1, a)
+ *
+ */
+
+static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyArray_MultiIterNew1(PyObject *__pyx_v_a) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("PyArray_MultiIterNew1", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":735
+ *
+ * cdef inline object PyArray_MultiIterNew1(a):
+ * return PyArray_MultiIterNew(1, a) # <<<<<<<<<<<<<<
+ *
+ * cdef inline object PyArray_MultiIterNew2(a, b):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyArray_MultiIterNew(1, ((void *)__pyx_v_a)); if (unlikely(!__pyx_t_1)) __PYX_ERR(2, 735, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":734
+ * ctypedef npy_cdouble complex_t
+ *
+ * cdef inline object PyArray_MultiIterNew1(a): # <<<<<<<<<<<<<<
+ * return PyArray_MultiIterNew(1, a)
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("numpy.PyArray_MultiIterNew1", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":737
+ * return PyArray_MultiIterNew(1, a)
+ *
+ * cdef inline object PyArray_MultiIterNew2(a, b): # <<<<<<<<<<<<<<
+ * return PyArray_MultiIterNew(2, a, b)
+ *
+ */
+
+static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyArray_MultiIterNew2(PyObject *__pyx_v_a, PyObject *__pyx_v_b) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("PyArray_MultiIterNew2", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":738
+ *
+ * cdef inline object PyArray_MultiIterNew2(a, b):
+ * return PyArray_MultiIterNew(2, a, b) # <<<<<<<<<<<<<<
+ *
+ * cdef inline object PyArray_MultiIterNew3(a, b, c):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyArray_MultiIterNew(2, ((void *)__pyx_v_a), ((void *)__pyx_v_b)); if (unlikely(!__pyx_t_1)) __PYX_ERR(2, 738, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":737
+ * return PyArray_MultiIterNew(1, a)
+ *
+ * cdef inline object PyArray_MultiIterNew2(a, b): # <<<<<<<<<<<<<<
+ * return PyArray_MultiIterNew(2, a, b)
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("numpy.PyArray_MultiIterNew2", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":740
+ * return PyArray_MultiIterNew(2, a, b)
+ *
+ * cdef inline object PyArray_MultiIterNew3(a, b, c): # <<<<<<<<<<<<<<
+ * return PyArray_MultiIterNew(3, a, b, c)
+ *
+ */
+
+static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyArray_MultiIterNew3(PyObject *__pyx_v_a, PyObject *__pyx_v_b, PyObject *__pyx_v_c) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("PyArray_MultiIterNew3", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":741
+ *
+ * cdef inline object PyArray_MultiIterNew3(a, b, c):
+ * return PyArray_MultiIterNew(3, a, b, c) # <<<<<<<<<<<<<<
+ *
+ * cdef inline object PyArray_MultiIterNew4(a, b, c, d):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyArray_MultiIterNew(3, ((void *)__pyx_v_a), ((void *)__pyx_v_b), ((void *)__pyx_v_c)); if (unlikely(!__pyx_t_1)) __PYX_ERR(2, 741, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":740
+ * return PyArray_MultiIterNew(2, a, b)
+ *
+ * cdef inline object PyArray_MultiIterNew3(a, b, c): # <<<<<<<<<<<<<<
+ * return PyArray_MultiIterNew(3, a, b, c)
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("numpy.PyArray_MultiIterNew3", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":743
+ * return PyArray_MultiIterNew(3, a, b, c)
+ *
+ * cdef inline object PyArray_MultiIterNew4(a, b, c, d): # <<<<<<<<<<<<<<
+ * return PyArray_MultiIterNew(4, a, b, c, d)
+ *
+ */
+
+static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyArray_MultiIterNew4(PyObject *__pyx_v_a, PyObject *__pyx_v_b, PyObject *__pyx_v_c, PyObject *__pyx_v_d) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("PyArray_MultiIterNew4", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":744
+ *
+ * cdef inline object PyArray_MultiIterNew4(a, b, c, d):
+ * return PyArray_MultiIterNew(4, a, b, c, d) # <<<<<<<<<<<<<<
+ *
+ * cdef inline object PyArray_MultiIterNew5(a, b, c, d, e):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyArray_MultiIterNew(4, ((void *)__pyx_v_a), ((void *)__pyx_v_b), ((void *)__pyx_v_c), ((void *)__pyx_v_d)); if (unlikely(!__pyx_t_1)) __PYX_ERR(2, 744, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":743
+ * return PyArray_MultiIterNew(3, a, b, c)
+ *
+ * cdef inline object PyArray_MultiIterNew4(a, b, c, d): # <<<<<<<<<<<<<<
+ * return PyArray_MultiIterNew(4, a, b, c, d)
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("numpy.PyArray_MultiIterNew4", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":746
+ * return PyArray_MultiIterNew(4, a, b, c, d)
+ *
+ * cdef inline object PyArray_MultiIterNew5(a, b, c, d, e): # <<<<<<<<<<<<<<
+ * return PyArray_MultiIterNew(5, a, b, c, d, e)
+ *
+ */
+
+static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyArray_MultiIterNew5(PyObject *__pyx_v_a, PyObject *__pyx_v_b, PyObject *__pyx_v_c, PyObject *__pyx_v_d, PyObject *__pyx_v_e) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("PyArray_MultiIterNew5", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":747
+ *
+ * cdef inline object PyArray_MultiIterNew5(a, b, c, d, e):
+ * return PyArray_MultiIterNew(5, a, b, c, d, e) # <<<<<<<<<<<<<<
+ *
+ * cdef inline tuple PyDataType_SHAPE(dtype d):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyArray_MultiIterNew(5, ((void *)__pyx_v_a), ((void *)__pyx_v_b), ((void *)__pyx_v_c), ((void *)__pyx_v_d), ((void *)__pyx_v_e)); if (unlikely(!__pyx_t_1)) __PYX_ERR(2, 747, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":746
+ * return PyArray_MultiIterNew(4, a, b, c, d)
+ *
+ * cdef inline object PyArray_MultiIterNew5(a, b, c, d, e): # <<<<<<<<<<<<<<
+ * return PyArray_MultiIterNew(5, a, b, c, d, e)
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("numpy.PyArray_MultiIterNew5", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":749
+ * return PyArray_MultiIterNew(5, a, b, c, d, e)
+ *
+ * cdef inline tuple PyDataType_SHAPE(dtype d): # <<<<<<<<<<<<<<
+ * if PyDataType_HASSUBARRAY(d):
+ * return d.subarray.shape
+ */
+
+static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyDataType_SHAPE(PyArray_Descr *__pyx_v_d) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ __Pyx_RefNannySetupContext("PyDataType_SHAPE", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":750
+ *
+ * cdef inline tuple PyDataType_SHAPE(dtype d):
+ * if PyDataType_HASSUBARRAY(d): # <<<<<<<<<<<<<<
+ * return d.subarray.shape
+ * else:
+ */
+ __pyx_t_1 = (PyDataType_HASSUBARRAY(__pyx_v_d) != 0);
+ if (__pyx_t_1) {
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":751
+ * cdef inline tuple PyDataType_SHAPE(dtype d):
+ * if PyDataType_HASSUBARRAY(d):
+ * return d.subarray.shape # <<<<<<<<<<<<<<
+ * else:
+ * return ()
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));
+ __pyx_r = ((PyObject*)__pyx_v_d->subarray->shape);
+ goto __pyx_L0;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":750
+ *
+ * cdef inline tuple PyDataType_SHAPE(dtype d):
+ * if PyDataType_HASSUBARRAY(d): # <<<<<<<<<<<<<<
+ * return d.subarray.shape
+ * else:
+ */
+ }
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":753
+ * return d.subarray.shape
+ * else:
+ * return () # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ /*else*/ {
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(__pyx_empty_tuple);
+ __pyx_r = __pyx_empty_tuple;
+ goto __pyx_L0;
+ }
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":749
+ * return PyArray_MultiIterNew(5, a, b, c, d, e)
+ *
+ * cdef inline tuple PyDataType_SHAPE(dtype d): # <<<<<<<<<<<<<<
+ * if PyDataType_HASSUBARRAY(d):
+ * return d.subarray.shape
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":868
+ * int _import_umath() except -1
+ *
+ * cdef inline void set_array_base(ndarray arr, object base): # <<<<<<<<<<<<<<
+ * Py_INCREF(base) # important to do this before stealing the reference below!
+ * PyArray_SetBaseObject(arr, base)
+ */
+
+static CYTHON_INLINE void __pyx_f_5numpy_set_array_base(PyArrayObject *__pyx_v_arr, PyObject *__pyx_v_base) {
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("set_array_base", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":869
+ *
+ * cdef inline void set_array_base(ndarray arr, object base):
+ * Py_INCREF(base) # important to do this before stealing the reference below! # <<<<<<<<<<<<<<
+ * PyArray_SetBaseObject(arr, base)
+ *
+ */
+ Py_INCREF(__pyx_v_base);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":870
+ * cdef inline void set_array_base(ndarray arr, object base):
+ * Py_INCREF(base) # important to do this before stealing the reference below!
+ * PyArray_SetBaseObject(arr, base) # <<<<<<<<<<<<<<
+ *
+ * cdef inline object get_array_base(ndarray arr):
+ */
+ (void)(PyArray_SetBaseObject(__pyx_v_arr, __pyx_v_base));
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":868
+ * int _import_umath() except -1
+ *
+ * cdef inline void set_array_base(ndarray arr, object base): # <<<<<<<<<<<<<<
+ * Py_INCREF(base) # important to do this before stealing the reference below!
+ * PyArray_SetBaseObject(arr, base)
+ */
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":872
+ * PyArray_SetBaseObject(arr, base)
+ *
+ * cdef inline object get_array_base(ndarray arr): # <<<<<<<<<<<<<<
+ * base = PyArray_BASE(arr)
+ * if base is NULL:
+ */
+
+static CYTHON_INLINE PyObject *__pyx_f_5numpy_get_array_base(PyArrayObject *__pyx_v_arr) {
+ PyObject *__pyx_v_base;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ __Pyx_RefNannySetupContext("get_array_base", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":873
+ *
+ * cdef inline object get_array_base(ndarray arr):
+ * base = PyArray_BASE(arr) # <<<<<<<<<<<<<<
+ * if base is NULL:
+ * return None
+ */
+ __pyx_v_base = PyArray_BASE(__pyx_v_arr);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":874
+ * cdef inline object get_array_base(ndarray arr):
+ * base = PyArray_BASE(arr)
+ * if base is NULL: # <<<<<<<<<<<<<<
+ * return None
+ * return base
+ */
+ __pyx_t_1 = ((__pyx_v_base == NULL) != 0);
+ if (__pyx_t_1) {
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":875
+ * base = PyArray_BASE(arr)
+ * if base is NULL:
+ * return None # <<<<<<<<<<<<<<
+ * return base
+ *
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":874
+ * cdef inline object get_array_base(ndarray arr):
+ * base = PyArray_BASE(arr)
+ * if base is NULL: # <<<<<<<<<<<<<<
+ * return None
+ * return base
+ */
+ }
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":876
+ * if base is NULL:
+ * return None
+ * return base # <<<<<<<<<<<<<<
+ *
+ * # Versions of the import_* functions which are more suitable for
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(((PyObject *)__pyx_v_base));
+ __pyx_r = ((PyObject *)__pyx_v_base);
+ goto __pyx_L0;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":872
+ * PyArray_SetBaseObject(arr, base)
+ *
+ * cdef inline object get_array_base(ndarray arr): # <<<<<<<<<<<<<<
+ * base = PyArray_BASE(arr)
+ * if base is NULL:
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":880
+ * # Versions of the import_* functions which are more suitable for
+ * # Cython code.
+ * cdef inline int import_array() except -1: # <<<<<<<<<<<<<<
+ * try:
+ * __pyx_import_array()
+ */
+
+static CYTHON_INLINE int __pyx_f_5numpy_import_array(void) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_t_4;
+ PyObject *__pyx_t_5 = NULL;
+ PyObject *__pyx_t_6 = NULL;
+ PyObject *__pyx_t_7 = NULL;
+ PyObject *__pyx_t_8 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("import_array", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":881
+ * # Cython code.
+ * cdef inline int import_array() except -1:
+ * try: # <<<<<<<<<<<<<<
+ * __pyx_import_array()
+ * except Exception:
+ */
+ {
+ __Pyx_PyThreadState_declare
+ __Pyx_PyThreadState_assign
+ __Pyx_ExceptionSave(&__pyx_t_1, &__pyx_t_2, &__pyx_t_3);
+ __Pyx_XGOTREF(__pyx_t_1);
+ __Pyx_XGOTREF(__pyx_t_2);
+ __Pyx_XGOTREF(__pyx_t_3);
+ /*try:*/ {
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":882
+ * cdef inline int import_array() except -1:
+ * try:
+ * __pyx_import_array() # <<<<<<<<<<<<<<
+ * except Exception:
+ * raise ImportError("numpy.core.multiarray failed to import")
+ */
+ __pyx_t_4 = _import_array(); if (unlikely(__pyx_t_4 == ((int)-1))) __PYX_ERR(2, 882, __pyx_L3_error)
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":881
+ * # Cython code.
+ * cdef inline int import_array() except -1:
+ * try: # <<<<<<<<<<<<<<
+ * __pyx_import_array()
+ * except Exception:
+ */
+ }
+ __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0;
+ goto __pyx_L8_try_end;
+ __pyx_L3_error:;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":883
+ * try:
+ * __pyx_import_array()
+ * except Exception: # <<<<<<<<<<<<<<
+ * raise ImportError("numpy.core.multiarray failed to import")
+ *
+ */
+ __pyx_t_4 = __Pyx_PyErr_ExceptionMatches(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])));
+ if (__pyx_t_4) {
+ __Pyx_AddTraceback("numpy.import_array", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ if (__Pyx_GetException(&__pyx_t_5, &__pyx_t_6, &__pyx_t_7) < 0) __PYX_ERR(2, 883, __pyx_L5_except_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_GOTREF(__pyx_t_6);
+ __Pyx_GOTREF(__pyx_t_7);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":884
+ * __pyx_import_array()
+ * except Exception:
+ * raise ImportError("numpy.core.multiarray failed to import") # <<<<<<<<<<<<<<
+ *
+ * cdef inline int import_umath() except -1:
+ */
+ __pyx_t_8 = __Pyx_PyObject_Call(__pyx_builtin_ImportError, __pyx_tuple__3, NULL); if (unlikely(!__pyx_t_8)) __PYX_ERR(2, 884, __pyx_L5_except_error)
+ __Pyx_GOTREF(__pyx_t_8);
+ __Pyx_Raise(__pyx_t_8, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
+ __PYX_ERR(2, 884, __pyx_L5_except_error)
+ }
+ goto __pyx_L5_except_error;
+ __pyx_L5_except_error:;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":881
+ * # Cython code.
+ * cdef inline int import_array() except -1:
+ * try: # <<<<<<<<<<<<<<
+ * __pyx_import_array()
+ * except Exception:
+ */
+ __Pyx_XGIVEREF(__pyx_t_1);
+ __Pyx_XGIVEREF(__pyx_t_2);
+ __Pyx_XGIVEREF(__pyx_t_3);
+ __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3);
+ goto __pyx_L1_error;
+ __pyx_L8_try_end:;
+ }
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":880
+ * # Versions of the import_* functions which are more suitable for
+ * # Cython code.
+ * cdef inline int import_array() except -1: # <<<<<<<<<<<<<<
+ * try:
+ * __pyx_import_array()
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_XDECREF(__pyx_t_6);
+ __Pyx_XDECREF(__pyx_t_7);
+ __Pyx_XDECREF(__pyx_t_8);
+ __Pyx_AddTraceback("numpy.import_array", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = -1;
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":886
+ * raise ImportError("numpy.core.multiarray failed to import")
+ *
+ * cdef inline int import_umath() except -1: # <<<<<<<<<<<<<<
+ * try:
+ * _import_umath()
+ */
+
+static CYTHON_INLINE int __pyx_f_5numpy_import_umath(void) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_t_4;
+ PyObject *__pyx_t_5 = NULL;
+ PyObject *__pyx_t_6 = NULL;
+ PyObject *__pyx_t_7 = NULL;
+ PyObject *__pyx_t_8 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("import_umath", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":887
+ *
+ * cdef inline int import_umath() except -1:
+ * try: # <<<<<<<<<<<<<<
+ * _import_umath()
+ * except Exception:
+ */
+ {
+ __Pyx_PyThreadState_declare
+ __Pyx_PyThreadState_assign
+ __Pyx_ExceptionSave(&__pyx_t_1, &__pyx_t_2, &__pyx_t_3);
+ __Pyx_XGOTREF(__pyx_t_1);
+ __Pyx_XGOTREF(__pyx_t_2);
+ __Pyx_XGOTREF(__pyx_t_3);
+ /*try:*/ {
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":888
+ * cdef inline int import_umath() except -1:
+ * try:
+ * _import_umath() # <<<<<<<<<<<<<<
+ * except Exception:
+ * raise ImportError("numpy.core.umath failed to import")
+ */
+ __pyx_t_4 = _import_umath(); if (unlikely(__pyx_t_4 == ((int)-1))) __PYX_ERR(2, 888, __pyx_L3_error)
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":887
+ *
+ * cdef inline int import_umath() except -1:
+ * try: # <<<<<<<<<<<<<<
+ * _import_umath()
+ * except Exception:
+ */
+ }
+ __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0;
+ goto __pyx_L8_try_end;
+ __pyx_L3_error:;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":889
+ * try:
+ * _import_umath()
+ * except Exception: # <<<<<<<<<<<<<<
+ * raise ImportError("numpy.core.umath failed to import")
+ *
+ */
+ __pyx_t_4 = __Pyx_PyErr_ExceptionMatches(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])));
+ if (__pyx_t_4) {
+ __Pyx_AddTraceback("numpy.import_umath", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ if (__Pyx_GetException(&__pyx_t_5, &__pyx_t_6, &__pyx_t_7) < 0) __PYX_ERR(2, 889, __pyx_L5_except_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_GOTREF(__pyx_t_6);
+ __Pyx_GOTREF(__pyx_t_7);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":890
+ * _import_umath()
+ * except Exception:
+ * raise ImportError("numpy.core.umath failed to import") # <<<<<<<<<<<<<<
+ *
+ * cdef inline int import_ufunc() except -1:
+ */
+ __pyx_t_8 = __Pyx_PyObject_Call(__pyx_builtin_ImportError, __pyx_tuple__4, NULL); if (unlikely(!__pyx_t_8)) __PYX_ERR(2, 890, __pyx_L5_except_error)
+ __Pyx_GOTREF(__pyx_t_8);
+ __Pyx_Raise(__pyx_t_8, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
+ __PYX_ERR(2, 890, __pyx_L5_except_error)
+ }
+ goto __pyx_L5_except_error;
+ __pyx_L5_except_error:;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":887
+ *
+ * cdef inline int import_umath() except -1:
+ * try: # <<<<<<<<<<<<<<
+ * _import_umath()
+ * except Exception:
+ */
+ __Pyx_XGIVEREF(__pyx_t_1);
+ __Pyx_XGIVEREF(__pyx_t_2);
+ __Pyx_XGIVEREF(__pyx_t_3);
+ __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3);
+ goto __pyx_L1_error;
+ __pyx_L8_try_end:;
+ }
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":886
+ * raise ImportError("numpy.core.multiarray failed to import")
+ *
+ * cdef inline int import_umath() except -1: # <<<<<<<<<<<<<<
+ * try:
+ * _import_umath()
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_XDECREF(__pyx_t_6);
+ __Pyx_XDECREF(__pyx_t_7);
+ __Pyx_XDECREF(__pyx_t_8);
+ __Pyx_AddTraceback("numpy.import_umath", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = -1;
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":892
+ * raise ImportError("numpy.core.umath failed to import")
+ *
+ * cdef inline int import_ufunc() except -1: # <<<<<<<<<<<<<<
+ * try:
+ * _import_umath()
+ */
+
+static CYTHON_INLINE int __pyx_f_5numpy_import_ufunc(void) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_t_4;
+ PyObject *__pyx_t_5 = NULL;
+ PyObject *__pyx_t_6 = NULL;
+ PyObject *__pyx_t_7 = NULL;
+ PyObject *__pyx_t_8 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("import_ufunc", 0);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":893
+ *
+ * cdef inline int import_ufunc() except -1:
+ * try: # <<<<<<<<<<<<<<
+ * _import_umath()
+ * except Exception:
+ */
+ {
+ __Pyx_PyThreadState_declare
+ __Pyx_PyThreadState_assign
+ __Pyx_ExceptionSave(&__pyx_t_1, &__pyx_t_2, &__pyx_t_3);
+ __Pyx_XGOTREF(__pyx_t_1);
+ __Pyx_XGOTREF(__pyx_t_2);
+ __Pyx_XGOTREF(__pyx_t_3);
+ /*try:*/ {
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":894
+ * cdef inline int import_ufunc() except -1:
+ * try:
+ * _import_umath() # <<<<<<<<<<<<<<
+ * except Exception:
+ * raise ImportError("numpy.core.umath failed to import")
+ */
+ __pyx_t_4 = _import_umath(); if (unlikely(__pyx_t_4 == ((int)-1))) __PYX_ERR(2, 894, __pyx_L3_error)
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":893
+ *
+ * cdef inline int import_ufunc() except -1:
+ * try: # <<<<<<<<<<<<<<
+ * _import_umath()
+ * except Exception:
+ */
+ }
+ __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0;
+ goto __pyx_L8_try_end;
+ __pyx_L3_error:;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":895
+ * try:
+ * _import_umath()
+ * except Exception: # <<<<<<<<<<<<<<
+ * raise ImportError("numpy.core.umath failed to import")
+ *
+ */
+ __pyx_t_4 = __Pyx_PyErr_ExceptionMatches(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])));
+ if (__pyx_t_4) {
+ __Pyx_AddTraceback("numpy.import_ufunc", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ if (__Pyx_GetException(&__pyx_t_5, &__pyx_t_6, &__pyx_t_7) < 0) __PYX_ERR(2, 895, __pyx_L5_except_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_GOTREF(__pyx_t_6);
+ __Pyx_GOTREF(__pyx_t_7);
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":896
+ * _import_umath()
+ * except Exception:
+ * raise ImportError("numpy.core.umath failed to import") # <<<<<<<<<<<<<<
+ *
+ * cdef extern from *:
+ */
+ __pyx_t_8 = __Pyx_PyObject_Call(__pyx_builtin_ImportError, __pyx_tuple__4, NULL); if (unlikely(!__pyx_t_8)) __PYX_ERR(2, 896, __pyx_L5_except_error)
+ __Pyx_GOTREF(__pyx_t_8);
+ __Pyx_Raise(__pyx_t_8, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
+ __PYX_ERR(2, 896, __pyx_L5_except_error)
+ }
+ goto __pyx_L5_except_error;
+ __pyx_L5_except_error:;
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":893
+ *
+ * cdef inline int import_ufunc() except -1:
+ * try: # <<<<<<<<<<<<<<
+ * _import_umath()
+ * except Exception:
+ */
+ __Pyx_XGIVEREF(__pyx_t_1);
+ __Pyx_XGIVEREF(__pyx_t_2);
+ __Pyx_XGIVEREF(__pyx_t_3);
+ __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3);
+ goto __pyx_L1_error;
+ __pyx_L8_try_end:;
+ }
+
+ /* "../../../../anaconda3/envs/mmaction/lib/python3.7/site-packages/numpy/__init__.pxd":892
+ * raise ImportError("numpy.core.umath failed to import")
+ *
+ * cdef inline int import_ufunc() except -1: # <<<<<<<<<<<<<<
+ * try:
+ * _import_umath()
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_XDECREF(__pyx_t_6);
+ __Pyx_XDECREF(__pyx_t_7);
+ __Pyx_XDECREF(__pyx_t_8);
+ __Pyx_AddTraceback("numpy.import_ufunc", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = -1;
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "string.to_py":31
+ *
+ * @cname("__pyx_convert_PyObject_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyObject_string_to_py_std__in_string(const string& s): # <<<<<<<<<<<<<<
+ * return __Pyx_PyObject_FromStringAndSize(s.data(), s.size())
+ * cdef extern from *:
+ */
+
+static CYTHON_INLINE PyObject *__pyx_convert_PyObject_string_to_py_std__in_string(std::string const &__pyx_v_s) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__pyx_convert_PyObject_string_to_py_std__in_string", 0);
+
+ /* "string.to_py":32
+ * @cname("__pyx_convert_PyObject_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyObject_string_to_py_std__in_string(const string& s):
+ * return __Pyx_PyObject_FromStringAndSize(s.data(), s.size()) # <<<<<<<<<<<<<<
+ * cdef extern from *:
+ * cdef object __Pyx_PyUnicode_FromStringAndSize(const char*, size_t)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyObject_FromStringAndSize(__pyx_v_s.data(), __pyx_v_s.size()); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 32, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "string.to_py":31
+ *
+ * @cname("__pyx_convert_PyObject_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyObject_string_to_py_std__in_string(const string& s): # <<<<<<<<<<<<<<
+ * return __Pyx_PyObject_FromStringAndSize(s.data(), s.size())
+ * cdef extern from *:
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("string.to_py.__pyx_convert_PyObject_string_to_py_std__in_string", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "string.to_py":37
+ *
+ * @cname("__pyx_convert_PyUnicode_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyUnicode_string_to_py_std__in_string(const string& s): # <<<<<<<<<<<<<<
+ * return __Pyx_PyUnicode_FromStringAndSize(s.data(), s.size())
+ * cdef extern from *:
+ */
+
+static CYTHON_INLINE PyObject *__pyx_convert_PyUnicode_string_to_py_std__in_string(std::string const &__pyx_v_s) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__pyx_convert_PyUnicode_string_to_py_std__in_string", 0);
+
+ /* "string.to_py":38
+ * @cname("__pyx_convert_PyUnicode_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyUnicode_string_to_py_std__in_string(const string& s):
+ * return __Pyx_PyUnicode_FromStringAndSize(s.data(), s.size()) # <<<<<<<<<<<<<<
+ * cdef extern from *:
+ * cdef object __Pyx_PyStr_FromStringAndSize(const char*, size_t)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyUnicode_FromStringAndSize(__pyx_v_s.data(), __pyx_v_s.size()); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 38, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "string.to_py":37
+ *
+ * @cname("__pyx_convert_PyUnicode_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyUnicode_string_to_py_std__in_string(const string& s): # <<<<<<<<<<<<<<
+ * return __Pyx_PyUnicode_FromStringAndSize(s.data(), s.size())
+ * cdef extern from *:
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("string.to_py.__pyx_convert_PyUnicode_string_to_py_std__in_string", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "string.to_py":43
+ *
+ * @cname("__pyx_convert_PyStr_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyStr_string_to_py_std__in_string(const string& s): # <<<<<<<<<<<<<<
+ * return __Pyx_PyStr_FromStringAndSize(s.data(), s.size())
+ * cdef extern from *:
+ */
+
+static CYTHON_INLINE PyObject *__pyx_convert_PyStr_string_to_py_std__in_string(std::string const &__pyx_v_s) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__pyx_convert_PyStr_string_to_py_std__in_string", 0);
+
+ /* "string.to_py":44
+ * @cname("__pyx_convert_PyStr_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyStr_string_to_py_std__in_string(const string& s):
+ * return __Pyx_PyStr_FromStringAndSize(s.data(), s.size()) # <<<<<<<<<<<<<<
+ * cdef extern from *:
+ * cdef object __Pyx_PyBytes_FromStringAndSize(const char*, size_t)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyStr_FromStringAndSize(__pyx_v_s.data(), __pyx_v_s.size()); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 44, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "string.to_py":43
+ *
+ * @cname("__pyx_convert_PyStr_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyStr_string_to_py_std__in_string(const string& s): # <<<<<<<<<<<<<<
+ * return __Pyx_PyStr_FromStringAndSize(s.data(), s.size())
+ * cdef extern from *:
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("string.to_py.__pyx_convert_PyStr_string_to_py_std__in_string", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "string.to_py":49
+ *
+ * @cname("__pyx_convert_PyBytes_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyBytes_string_to_py_std__in_string(const string& s): # <<<<<<<<<<<<<<
+ * return __Pyx_PyBytes_FromStringAndSize(s.data(), s.size())
+ * cdef extern from *:
+ */
+
+static CYTHON_INLINE PyObject *__pyx_convert_PyBytes_string_to_py_std__in_string(std::string const &__pyx_v_s) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__pyx_convert_PyBytes_string_to_py_std__in_string", 0);
+
+ /* "string.to_py":50
+ * @cname("__pyx_convert_PyBytes_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyBytes_string_to_py_std__in_string(const string& s):
+ * return __Pyx_PyBytes_FromStringAndSize(s.data(), s.size()) # <<<<<<<<<<<<<<
+ * cdef extern from *:
+ * cdef object __Pyx_PyByteArray_FromStringAndSize(const char*, size_t)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyBytes_FromStringAndSize(__pyx_v_s.data(), __pyx_v_s.size()); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 50, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "string.to_py":49
+ *
+ * @cname("__pyx_convert_PyBytes_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyBytes_string_to_py_std__in_string(const string& s): # <<<<<<<<<<<<<<
+ * return __Pyx_PyBytes_FromStringAndSize(s.data(), s.size())
+ * cdef extern from *:
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("string.to_py.__pyx_convert_PyBytes_string_to_py_std__in_string", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "string.to_py":55
+ *
+ * @cname("__pyx_convert_PyByteArray_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyByteArray_string_to_py_std__in_string(const string& s): # <<<<<<<<<<<<<<
+ * return __Pyx_PyByteArray_FromStringAndSize(s.data(), s.size())
+ *
+ */
+
+static CYTHON_INLINE PyObject *__pyx_convert_PyByteArray_string_to_py_std__in_string(std::string const &__pyx_v_s) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__pyx_convert_PyByteArray_string_to_py_std__in_string", 0);
+
+ /* "string.to_py":56
+ * @cname("__pyx_convert_PyByteArray_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyByteArray_string_to_py_std__in_string(const string& s):
+ * return __Pyx_PyByteArray_FromStringAndSize(s.data(), s.size()) # <<<<<<<<<<<<<<
+ *
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyByteArray_FromStringAndSize(__pyx_v_s.data(), __pyx_v_s.size()); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 56, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "string.to_py":55
+ *
+ * @cname("__pyx_convert_PyByteArray_string_to_py_std__in_string")
+ * cdef inline object __pyx_convert_PyByteArray_string_to_py_std__in_string(const string& s): # <<<<<<<<<<<<<<
+ * return __Pyx_PyByteArray_FromStringAndSize(s.data(), s.size())
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("string.to_py.__pyx_convert_PyByteArray_string_to_py_std__in_string", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "string.from_py":13
+ *
+ * @cname("__pyx_convert_string_from_py_std__in_string")
+ * cdef string __pyx_convert_string_from_py_std__in_string(object o) except *: # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t length = 0
+ * cdef const char* data = __Pyx_PyObject_AsStringAndSize(o, &length)
+ */
+
+static std::string __pyx_convert_string_from_py_std__in_string(PyObject *__pyx_v_o) {
+ Py_ssize_t __pyx_v_length;
+ char const *__pyx_v_data;
+ std::string __pyx_r;
+ __Pyx_RefNannyDeclarations
+ char const *__pyx_t_1;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__pyx_convert_string_from_py_std__in_string", 0);
+
+ /* "string.from_py":14
+ * @cname("__pyx_convert_string_from_py_std__in_string")
+ * cdef string __pyx_convert_string_from_py_std__in_string(object o) except *:
+ * cdef Py_ssize_t length = 0 # <<<<<<<<<<<<<<
+ * cdef const char* data = __Pyx_PyObject_AsStringAndSize(o, &length)
+ * return string(data, length)
+ */
+ __pyx_v_length = 0;
+
+ /* "string.from_py":15
+ * cdef string __pyx_convert_string_from_py_std__in_string(object o) except *:
+ * cdef Py_ssize_t length = 0
+ * cdef const char* data = __Pyx_PyObject_AsStringAndSize(o, &length) # <<<<<<<<<<<<<<
+ * return string(data, length)
+ *
+ */
+ __pyx_t_1 = __Pyx_PyObject_AsStringAndSize(__pyx_v_o, (&__pyx_v_length)); if (unlikely(__pyx_t_1 == ((char const *)NULL))) __PYX_ERR(1, 15, __pyx_L1_error)
+ __pyx_v_data = __pyx_t_1;
+
+ /* "string.from_py":16
+ * cdef Py_ssize_t length = 0
+ * cdef const char* data = __Pyx_PyObject_AsStringAndSize(o, &length)
+ * return string(data, length) # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_r = std::string(__pyx_v_data, __pyx_v_length);
+ goto __pyx_L0;
+
+ /* "string.from_py":13
+ *
+ * @cname("__pyx_convert_string_from_py_std__in_string")
+ * cdef string __pyx_convert_string_from_py_std__in_string(object o) except *: # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t length = 0
+ * cdef const char* data = __Pyx_PyObject_AsStringAndSize(o, &length)
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_AddTraceback("string.from_py.__pyx_convert_string_from_py_std__in_string", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_pretend_to_initialize(&__pyx_r);
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":122
+ * cdef bint dtype_is_object
+ *
+ * def __cinit__(array self, tuple shape, Py_ssize_t itemsize, format not None, # <<<<<<<<<<<<<<
+ * mode="c", bint allocate_buffer=True):
+ *
+ */
+
+/* Python wrapper */
+static int __pyx_array___cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
+static int __pyx_array___cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
+ PyObject *__pyx_v_shape = 0;
+ Py_ssize_t __pyx_v_itemsize;
+ PyObject *__pyx_v_format = 0;
+ PyObject *__pyx_v_mode = 0;
+ int __pyx_v_allocate_buffer;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__cinit__ (wrapper)", 0);
+ {
+ static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_shape,&__pyx_n_s_itemsize,&__pyx_n_s_format,&__pyx_n_s_mode,&__pyx_n_s_allocate_buffer,0};
+ PyObject* values[5] = {0,0,0,0,0};
+ values[3] = ((PyObject *)__pyx_n_s_c);
+ if (unlikely(__pyx_kwds)) {
+ Py_ssize_t kw_args;
+ const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
+ switch (pos_args) {
+ case 5: values[4] = PyTuple_GET_ITEM(__pyx_args, 4);
+ CYTHON_FALLTHROUGH;
+ case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3);
+ CYTHON_FALLTHROUGH;
+ case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
+ CYTHON_FALLTHROUGH;
+ case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ CYTHON_FALLTHROUGH;
+ case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ CYTHON_FALLTHROUGH;
+ case 0: break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ kw_args = PyDict_Size(__pyx_kwds);
+ switch (pos_args) {
+ case 0:
+ if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_shape)) != 0)) kw_args--;
+ else goto __pyx_L5_argtuple_error;
+ CYTHON_FALLTHROUGH;
+ case 1:
+ if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_itemsize)) != 0)) kw_args--;
+ else {
+ __Pyx_RaiseArgtupleInvalid("__cinit__", 0, 3, 5, 1); __PYX_ERR(1, 122, __pyx_L3_error)
+ }
+ CYTHON_FALLTHROUGH;
+ case 2:
+ if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_format)) != 0)) kw_args--;
+ else {
+ __Pyx_RaiseArgtupleInvalid("__cinit__", 0, 3, 5, 2); __PYX_ERR(1, 122, __pyx_L3_error)
+ }
+ CYTHON_FALLTHROUGH;
+ case 3:
+ if (kw_args > 0) {
+ PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_mode);
+ if (value) { values[3] = value; kw_args--; }
+ }
+ CYTHON_FALLTHROUGH;
+ case 4:
+ if (kw_args > 0) {
+ PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_allocate_buffer);
+ if (value) { values[4] = value; kw_args--; }
+ }
+ }
+ if (unlikely(kw_args > 0)) {
+ if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "__cinit__") < 0)) __PYX_ERR(1, 122, __pyx_L3_error)
+ }
+ } else {
+ switch (PyTuple_GET_SIZE(__pyx_args)) {
+ case 5: values[4] = PyTuple_GET_ITEM(__pyx_args, 4);
+ CYTHON_FALLTHROUGH;
+ case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3);
+ CYTHON_FALLTHROUGH;
+ case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
+ values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ }
+ __pyx_v_shape = ((PyObject*)values[0]);
+ __pyx_v_itemsize = __Pyx_PyIndex_AsSsize_t(values[1]); if (unlikely((__pyx_v_itemsize == (Py_ssize_t)-1) && PyErr_Occurred())) __PYX_ERR(1, 122, __pyx_L3_error)
+ __pyx_v_format = values[2];
+ __pyx_v_mode = values[3];
+ if (values[4]) {
+ __pyx_v_allocate_buffer = __Pyx_PyObject_IsTrue(values[4]); if (unlikely((__pyx_v_allocate_buffer == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 123, __pyx_L3_error)
+ } else {
+
+ /* "View.MemoryView":123
+ *
+ * def __cinit__(array self, tuple shape, Py_ssize_t itemsize, format not None,
+ * mode="c", bint allocate_buffer=True): # <<<<<<<<<<<<<<
+ *
+ * cdef int idx
+ */
+ __pyx_v_allocate_buffer = ((int)1);
+ }
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L5_argtuple_error:;
+ __Pyx_RaiseArgtupleInvalid("__cinit__", 0, 3, 5, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(1, 122, __pyx_L3_error)
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("View.MemoryView.array.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return -1;
+ __pyx_L4_argument_unpacking_done:;
+ if (unlikely(!__Pyx_ArgTypeTest(((PyObject *)__pyx_v_shape), (&PyTuple_Type), 1, "shape", 1))) __PYX_ERR(1, 122, __pyx_L1_error)
+ if (unlikely(((PyObject *)__pyx_v_format) == Py_None)) {
+ PyErr_Format(PyExc_TypeError, "Argument '%.200s' must not be None", "format"); __PYX_ERR(1, 122, __pyx_L1_error)
+ }
+ __pyx_r = __pyx_array___pyx_pf_15View_dot_MemoryView_5array___cinit__(((struct __pyx_array_obj *)__pyx_v_self), __pyx_v_shape, __pyx_v_itemsize, __pyx_v_format, __pyx_v_mode, __pyx_v_allocate_buffer);
+
+ /* "View.MemoryView":122
+ * cdef bint dtype_is_object
+ *
+ * def __cinit__(array self, tuple shape, Py_ssize_t itemsize, format not None, # <<<<<<<<<<<<<<
+ * mode="c", bint allocate_buffer=True):
+ *
+ */
+
+ /* function exit code */
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __pyx_r = -1;
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_array___pyx_pf_15View_dot_MemoryView_5array___cinit__(struct __pyx_array_obj *__pyx_v_self, PyObject *__pyx_v_shape, Py_ssize_t __pyx_v_itemsize, PyObject *__pyx_v_format, PyObject *__pyx_v_mode, int __pyx_v_allocate_buffer) {
+ int __pyx_v_idx;
+ Py_ssize_t __pyx_v_i;
+ Py_ssize_t __pyx_v_dim;
+ PyObject **__pyx_v_p;
+ char __pyx_v_order;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ Py_ssize_t __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_t_4;
+ PyObject *__pyx_t_5 = NULL;
+ PyObject *__pyx_t_6 = NULL;
+ char *__pyx_t_7;
+ int __pyx_t_8;
+ Py_ssize_t __pyx_t_9;
+ PyObject *__pyx_t_10 = NULL;
+ Py_ssize_t __pyx_t_11;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__cinit__", 0);
+ __Pyx_INCREF(__pyx_v_format);
+
+ /* "View.MemoryView":129
+ * cdef PyObject **p
+ *
+ * self.ndim = len(shape) # <<<<<<<<<<<<<<
+ * self.itemsize = itemsize
+ *
+ */
+ if (unlikely(__pyx_v_shape == Py_None)) {
+ PyErr_SetString(PyExc_TypeError, "object of type 'NoneType' has no len()");
+ __PYX_ERR(1, 129, __pyx_L1_error)
+ }
+ __pyx_t_1 = PyTuple_GET_SIZE(__pyx_v_shape); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(1, 129, __pyx_L1_error)
+ __pyx_v_self->ndim = ((int)__pyx_t_1);
+
+ /* "View.MemoryView":130
+ *
+ * self.ndim = len(shape)
+ * self.itemsize = itemsize # <<<<<<<<<<<<<<
+ *
+ * if not self.ndim:
+ */
+ __pyx_v_self->itemsize = __pyx_v_itemsize;
+
+ /* "View.MemoryView":132
+ * self.itemsize = itemsize
+ *
+ * if not self.ndim: # <<<<<<<<<<<<<<
+ * raise ValueError("Empty shape tuple for cython.array")
+ *
+ */
+ __pyx_t_2 = ((!(__pyx_v_self->ndim != 0)) != 0);
+ if (unlikely(__pyx_t_2)) {
+
+ /* "View.MemoryView":133
+ *
+ * if not self.ndim:
+ * raise ValueError("Empty shape tuple for cython.array") # <<<<<<<<<<<<<<
+ *
+ * if itemsize <= 0:
+ */
+ __pyx_t_3 = __Pyx_PyObject_Call(__pyx_builtin_ValueError, __pyx_tuple__5, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 133, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_Raise(__pyx_t_3, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __PYX_ERR(1, 133, __pyx_L1_error)
+
+ /* "View.MemoryView":132
+ * self.itemsize = itemsize
+ *
+ * if not self.ndim: # <<<<<<<<<<<<<<
+ * raise ValueError("Empty shape tuple for cython.array")
+ *
+ */
+ }
+
+ /* "View.MemoryView":135
+ * raise ValueError("Empty shape tuple for cython.array")
+ *
+ * if itemsize <= 0: # <<<<<<<<<<<<<<
+ * raise ValueError("itemsize <= 0 for cython.array")
+ *
+ */
+ __pyx_t_2 = ((__pyx_v_itemsize <= 0) != 0);
+ if (unlikely(__pyx_t_2)) {
+
+ /* "View.MemoryView":136
+ *
+ * if itemsize <= 0:
+ * raise ValueError("itemsize <= 0 for cython.array") # <<<<<<<<<<<<<<
+ *
+ * if not isinstance(format, bytes):
+ */
+ __pyx_t_3 = __Pyx_PyObject_Call(__pyx_builtin_ValueError, __pyx_tuple__6, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 136, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_Raise(__pyx_t_3, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __PYX_ERR(1, 136, __pyx_L1_error)
+
+ /* "View.MemoryView":135
+ * raise ValueError("Empty shape tuple for cython.array")
+ *
+ * if itemsize <= 0: # <<<<<<<<<<<<<<
+ * raise ValueError("itemsize <= 0 for cython.array")
+ *
+ */
+ }
+
+ /* "View.MemoryView":138
+ * raise ValueError("itemsize <= 0 for cython.array")
+ *
+ * if not isinstance(format, bytes): # <<<<<<<<<<<<<<
+ * format = format.encode('ASCII')
+ * self._format = format # keep a reference to the byte string
+ */
+ __pyx_t_2 = PyBytes_Check(__pyx_v_format);
+ __pyx_t_4 = ((!(__pyx_t_2 != 0)) != 0);
+ if (__pyx_t_4) {
+
+ /* "View.MemoryView":139
+ *
+ * if not isinstance(format, bytes):
+ * format = format.encode('ASCII') # <<<<<<<<<<<<<<
+ * self._format = format # keep a reference to the byte string
+ * self.format = self._format
+ */
+ __pyx_t_5 = __Pyx_PyObject_GetAttrStr(__pyx_v_format, __pyx_n_s_encode); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 139, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __pyx_t_6 = NULL;
+ if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_5))) {
+ __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_5);
+ if (likely(__pyx_t_6)) {
+ PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_5);
+ __Pyx_INCREF(__pyx_t_6);
+ __Pyx_INCREF(function);
+ __Pyx_DECREF_SET(__pyx_t_5, function);
+ }
+ }
+ __pyx_t_3 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_5, __pyx_t_6, __pyx_n_s_ASCII) : __Pyx_PyObject_CallOneArg(__pyx_t_5, __pyx_n_s_ASCII);
+ __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0;
+ if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 139, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
+ __Pyx_DECREF_SET(__pyx_v_format, __pyx_t_3);
+ __pyx_t_3 = 0;
+
+ /* "View.MemoryView":138
+ * raise ValueError("itemsize <= 0 for cython.array")
+ *
+ * if not isinstance(format, bytes): # <<<<<<<<<<<<<<
+ * format = format.encode('ASCII')
+ * self._format = format # keep a reference to the byte string
+ */
+ }
+
+ /* "View.MemoryView":140
+ * if not isinstance(format, bytes):
+ * format = format.encode('ASCII')
+ * self._format = format # keep a reference to the byte string # <<<<<<<<<<<<<<
+ * self.format = self._format
+ *
+ */
+ if (!(likely(PyBytes_CheckExact(__pyx_v_format))||((__pyx_v_format) == Py_None)||(PyErr_Format(PyExc_TypeError, "Expected %.16s, got %.200s", "bytes", Py_TYPE(__pyx_v_format)->tp_name), 0))) __PYX_ERR(1, 140, __pyx_L1_error)
+ __pyx_t_3 = __pyx_v_format;
+ __Pyx_INCREF(__pyx_t_3);
+ __Pyx_GIVEREF(__pyx_t_3);
+ __Pyx_GOTREF(__pyx_v_self->_format);
+ __Pyx_DECREF(__pyx_v_self->_format);
+ __pyx_v_self->_format = ((PyObject*)__pyx_t_3);
+ __pyx_t_3 = 0;
+
+ /* "View.MemoryView":141
+ * format = format.encode('ASCII')
+ * self._format = format # keep a reference to the byte string
+ * self.format = self._format # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ if (unlikely(__pyx_v_self->_format == Py_None)) {
+ PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found");
+ __PYX_ERR(1, 141, __pyx_L1_error)
+ }
+ __pyx_t_7 = __Pyx_PyBytes_AsWritableString(__pyx_v_self->_format); if (unlikely((!__pyx_t_7) && PyErr_Occurred())) __PYX_ERR(1, 141, __pyx_L1_error)
+ __pyx_v_self->format = __pyx_t_7;
+
+ /* "View.MemoryView":144
+ *
+ *
+ * self._shape = PyObject_Malloc(sizeof(Py_ssize_t)*self.ndim*2) # <<<<<<<<<<<<<<
+ * self._strides = self._shape + self.ndim
+ *
+ */
+ __pyx_v_self->_shape = ((Py_ssize_t *)PyObject_Malloc((((sizeof(Py_ssize_t)) * __pyx_v_self->ndim) * 2)));
+
+ /* "View.MemoryView":145
+ *
+ * self._shape = PyObject_Malloc(sizeof(Py_ssize_t)*self.ndim*2)
+ * self._strides = self._shape + self.ndim # <<<<<<<<<<<<<<
+ *
+ * if not self._shape:
+ */
+ __pyx_v_self->_strides = (__pyx_v_self->_shape + __pyx_v_self->ndim);
+
+ /* "View.MemoryView":147
+ * self._strides = self._shape + self.ndim
+ *
+ * if not self._shape: # <<<<<<<<<<<<<<
+ * raise MemoryError("unable to allocate shape and strides.")
+ *
+ */
+ __pyx_t_4 = ((!(__pyx_v_self->_shape != 0)) != 0);
+ if (unlikely(__pyx_t_4)) {
+
+ /* "View.MemoryView":148
+ *
+ * if not self._shape:
+ * raise MemoryError("unable to allocate shape and strides.") # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_t_3 = __Pyx_PyObject_Call(__pyx_builtin_MemoryError, __pyx_tuple__7, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 148, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_Raise(__pyx_t_3, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __PYX_ERR(1, 148, __pyx_L1_error)
+
+ /* "View.MemoryView":147
+ * self._strides = self._shape + self.ndim
+ *
+ * if not self._shape: # <<<<<<<<<<<<<<
+ * raise MemoryError("unable to allocate shape and strides.")
+ *
+ */
+ }
+
+ /* "View.MemoryView":151
+ *
+ *
+ * for idx, dim in enumerate(shape): # <<<<<<<<<<<<<<
+ * if dim <= 0:
+ * raise ValueError("Invalid shape in axis %d: %d." % (idx, dim))
+ */
+ __pyx_t_8 = 0;
+ __pyx_t_3 = __pyx_v_shape; __Pyx_INCREF(__pyx_t_3); __pyx_t_1 = 0;
+ for (;;) {
+ if (__pyx_t_1 >= PyTuple_GET_SIZE(__pyx_t_3)) break;
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_5 = PyTuple_GET_ITEM(__pyx_t_3, __pyx_t_1); __Pyx_INCREF(__pyx_t_5); __pyx_t_1++; if (unlikely(0 < 0)) __PYX_ERR(1, 151, __pyx_L1_error)
+ #else
+ __pyx_t_5 = PySequence_ITEM(__pyx_t_3, __pyx_t_1); __pyx_t_1++; if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 151, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ #endif
+ __pyx_t_9 = __Pyx_PyIndex_AsSsize_t(__pyx_t_5); if (unlikely((__pyx_t_9 == (Py_ssize_t)-1) && PyErr_Occurred())) __PYX_ERR(1, 151, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
+ __pyx_v_dim = __pyx_t_9;
+ __pyx_v_idx = __pyx_t_8;
+ __pyx_t_8 = (__pyx_t_8 + 1);
+
+ /* "View.MemoryView":152
+ *
+ * for idx, dim in enumerate(shape):
+ * if dim <= 0: # <<<<<<<<<<<<<<
+ * raise ValueError("Invalid shape in axis %d: %d." % (idx, dim))
+ * self._shape[idx] = dim
+ */
+ __pyx_t_4 = ((__pyx_v_dim <= 0) != 0);
+ if (unlikely(__pyx_t_4)) {
+
+ /* "View.MemoryView":153
+ * for idx, dim in enumerate(shape):
+ * if dim <= 0:
+ * raise ValueError("Invalid shape in axis %d: %d." % (idx, dim)) # <<<<<<<<<<<<<<
+ * self._shape[idx] = dim
+ *
+ */
+ __pyx_t_5 = __Pyx_PyInt_From_int(__pyx_v_idx); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 153, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __pyx_t_6 = PyInt_FromSsize_t(__pyx_v_dim); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 153, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ __pyx_t_10 = PyTuple_New(2); if (unlikely(!__pyx_t_10)) __PYX_ERR(1, 153, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_10);
+ __Pyx_GIVEREF(__pyx_t_5);
+ PyTuple_SET_ITEM(__pyx_t_10, 0, __pyx_t_5);
+ __Pyx_GIVEREF(__pyx_t_6);
+ PyTuple_SET_ITEM(__pyx_t_10, 1, __pyx_t_6);
+ __pyx_t_5 = 0;
+ __pyx_t_6 = 0;
+ __pyx_t_6 = __Pyx_PyString_Format(__pyx_kp_s_Invalid_shape_in_axis_d_d, __pyx_t_10); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 153, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0;
+ __pyx_t_10 = __Pyx_PyObject_CallOneArg(__pyx_builtin_ValueError, __pyx_t_6); if (unlikely(!__pyx_t_10)) __PYX_ERR(1, 153, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_10);
+ __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
+ __Pyx_Raise(__pyx_t_10, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0;
+ __PYX_ERR(1, 153, __pyx_L1_error)
+
+ /* "View.MemoryView":152
+ *
+ * for idx, dim in enumerate(shape):
+ * if dim <= 0: # <<<<<<<<<<<<<<
+ * raise ValueError("Invalid shape in axis %d: %d." % (idx, dim))
+ * self._shape[idx] = dim
+ */
+ }
+
+ /* "View.MemoryView":154
+ * if dim <= 0:
+ * raise ValueError("Invalid shape in axis %d: %d." % (idx, dim))
+ * self._shape[idx] = dim # <<<<<<<<<<<<<<
+ *
+ * cdef char order
+ */
+ (__pyx_v_self->_shape[__pyx_v_idx]) = __pyx_v_dim;
+
+ /* "View.MemoryView":151
+ *
+ *
+ * for idx, dim in enumerate(shape): # <<<<<<<<<<<<<<
+ * if dim <= 0:
+ * raise ValueError("Invalid shape in axis %d: %d." % (idx, dim))
+ */
+ }
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+
+ /* "View.MemoryView":157
+ *
+ * cdef char order
+ * if mode == 'fortran': # <<<<<<<<<<<<<<
+ * order = b'F'
+ * self.mode = u'fortran'
+ */
+ __pyx_t_4 = (__Pyx_PyString_Equals(__pyx_v_mode, __pyx_n_s_fortran, Py_EQ)); if (unlikely(__pyx_t_4 < 0)) __PYX_ERR(1, 157, __pyx_L1_error)
+ if (__pyx_t_4) {
+
+ /* "View.MemoryView":158
+ * cdef char order
+ * if mode == 'fortran':
+ * order = b'F' # <<<<<<<<<<<<<<
+ * self.mode = u'fortran'
+ * elif mode == 'c':
+ */
+ __pyx_v_order = 'F';
+
+ /* "View.MemoryView":159
+ * if mode == 'fortran':
+ * order = b'F'
+ * self.mode = u'fortran' # <<<<<<<<<<<<<<
+ * elif mode == 'c':
+ * order = b'C'
+ */
+ __Pyx_INCREF(__pyx_n_u_fortran);
+ __Pyx_GIVEREF(__pyx_n_u_fortran);
+ __Pyx_GOTREF(__pyx_v_self->mode);
+ __Pyx_DECREF(__pyx_v_self->mode);
+ __pyx_v_self->mode = __pyx_n_u_fortran;
+
+ /* "View.MemoryView":157
+ *
+ * cdef char order
+ * if mode == 'fortran': # <<<<<<<<<<<<<<
+ * order = b'F'
+ * self.mode = u'fortran'
+ */
+ goto __pyx_L10;
+ }
+
+ /* "View.MemoryView":160
+ * order = b'F'
+ * self.mode = u'fortran'
+ * elif mode == 'c': # <<<<<<<<<<<<<<
+ * order = b'C'
+ * self.mode = u'c'
+ */
+ __pyx_t_4 = (__Pyx_PyString_Equals(__pyx_v_mode, __pyx_n_s_c, Py_EQ)); if (unlikely(__pyx_t_4 < 0)) __PYX_ERR(1, 160, __pyx_L1_error)
+ if (likely(__pyx_t_4)) {
+
+ /* "View.MemoryView":161
+ * self.mode = u'fortran'
+ * elif mode == 'c':
+ * order = b'C' # <<<<<<<<<<<<<<
+ * self.mode = u'c'
+ * else:
+ */
+ __pyx_v_order = 'C';
+
+ /* "View.MemoryView":162
+ * elif mode == 'c':
+ * order = b'C'
+ * self.mode = u'c' # <<<<<<<<<<<<<<
+ * else:
+ * raise ValueError("Invalid mode, expected 'c' or 'fortran', got %s" % mode)
+ */
+ __Pyx_INCREF(__pyx_n_u_c);
+ __Pyx_GIVEREF(__pyx_n_u_c);
+ __Pyx_GOTREF(__pyx_v_self->mode);
+ __Pyx_DECREF(__pyx_v_self->mode);
+ __pyx_v_self->mode = __pyx_n_u_c;
+
+ /* "View.MemoryView":160
+ * order = b'F'
+ * self.mode = u'fortran'
+ * elif mode == 'c': # <<<<<<<<<<<<<<
+ * order = b'C'
+ * self.mode = u'c'
+ */
+ goto __pyx_L10;
+ }
+
+ /* "View.MemoryView":164
+ * self.mode = u'c'
+ * else:
+ * raise ValueError("Invalid mode, expected 'c' or 'fortran', got %s" % mode) # <<<<<<<<<<<<<<
+ *
+ * self.len = fill_contig_strides_array(self._shape, self._strides,
+ */
+ /*else*/ {
+ __pyx_t_3 = __Pyx_PyString_FormatSafe(__pyx_kp_s_Invalid_mode_expected_c_or_fortr, __pyx_v_mode); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 164, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_t_10 = __Pyx_PyObject_CallOneArg(__pyx_builtin_ValueError, __pyx_t_3); if (unlikely(!__pyx_t_10)) __PYX_ERR(1, 164, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_10);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __Pyx_Raise(__pyx_t_10, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0;
+ __PYX_ERR(1, 164, __pyx_L1_error)
+ }
+ __pyx_L10:;
+
+ /* "View.MemoryView":166
+ * raise ValueError("Invalid mode, expected 'c' or 'fortran', got %s" % mode)
+ *
+ * self.len = fill_contig_strides_array(self._shape, self._strides, # <<<<<<<<<<<<<<
+ * itemsize, self.ndim, order)
+ *
+ */
+ __pyx_v_self->len = __pyx_fill_contig_strides_array(__pyx_v_self->_shape, __pyx_v_self->_strides, __pyx_v_itemsize, __pyx_v_self->ndim, __pyx_v_order);
+
+ /* "View.MemoryView":169
+ * itemsize, self.ndim, order)
+ *
+ * self.free_data = allocate_buffer # <<<<<<<<<<<<<<
+ * self.dtype_is_object = format == b'O'
+ * if allocate_buffer:
+ */
+ __pyx_v_self->free_data = __pyx_v_allocate_buffer;
+
+ /* "View.MemoryView":170
+ *
+ * self.free_data = allocate_buffer
+ * self.dtype_is_object = format == b'O' # <<<<<<<<<<<<<<
+ * if allocate_buffer:
+ *
+ */
+ __pyx_t_10 = PyObject_RichCompare(__pyx_v_format, __pyx_n_b_O, Py_EQ); __Pyx_XGOTREF(__pyx_t_10); if (unlikely(!__pyx_t_10)) __PYX_ERR(1, 170, __pyx_L1_error)
+ __pyx_t_4 = __Pyx_PyObject_IsTrue(__pyx_t_10); if (unlikely((__pyx_t_4 == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 170, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0;
+ __pyx_v_self->dtype_is_object = __pyx_t_4;
+
+ /* "View.MemoryView":171
+ * self.free_data = allocate_buffer
+ * self.dtype_is_object = format == b'O'
+ * if allocate_buffer: # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_t_4 = (__pyx_v_allocate_buffer != 0);
+ if (__pyx_t_4) {
+
+ /* "View.MemoryView":174
+ *
+ *
+ * self.data = malloc(self.len) # <<<<<<<<<<<<<<
+ * if not self.data:
+ * raise MemoryError("unable to allocate array data.")
+ */
+ __pyx_v_self->data = ((char *)malloc(__pyx_v_self->len));
+
+ /* "View.MemoryView":175
+ *
+ * self.data = malloc(self.len)
+ * if not self.data: # <<<<<<<<<<<<<<
+ * raise MemoryError("unable to allocate array data.")
+ *
+ */
+ __pyx_t_4 = ((!(__pyx_v_self->data != 0)) != 0);
+ if (unlikely(__pyx_t_4)) {
+
+ /* "View.MemoryView":176
+ * self.data = malloc(self.len)
+ * if not self.data:
+ * raise MemoryError("unable to allocate array data.") # <<<<<<<<<<<<<<
+ *
+ * if self.dtype_is_object:
+ */
+ __pyx_t_10 = __Pyx_PyObject_Call(__pyx_builtin_MemoryError, __pyx_tuple__8, NULL); if (unlikely(!__pyx_t_10)) __PYX_ERR(1, 176, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_10);
+ __Pyx_Raise(__pyx_t_10, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0;
+ __PYX_ERR(1, 176, __pyx_L1_error)
+
+ /* "View.MemoryView":175
+ *
+ * self.data = malloc(self.len)
+ * if not self.data: # <<<<<<<<<<<<<<
+ * raise MemoryError("unable to allocate array data.")
+ *
+ */
+ }
+
+ /* "View.MemoryView":178
+ * raise MemoryError("unable to allocate array data.")
+ *
+ * if self.dtype_is_object: # <<<<<<<<<<<<<<
+ * p = self.data
+ * for i in range(self.len / itemsize):
+ */
+ __pyx_t_4 = (__pyx_v_self->dtype_is_object != 0);
+ if (__pyx_t_4) {
+
+ /* "View.MemoryView":179
+ *
+ * if self.dtype_is_object:
+ * p = self.data # <<<<<<<<<<<<<<
+ * for i in range(self.len / itemsize):
+ * p[i] = Py_None
+ */
+ __pyx_v_p = ((PyObject **)__pyx_v_self->data);
+
+ /* "View.MemoryView":180
+ * if self.dtype_is_object:
+ * p = self.data
+ * for i in range(self.len / itemsize): # <<<<<<<<<<<<<<
+ * p[i] = Py_None
+ * Py_INCREF(Py_None)
+ */
+ if (unlikely(__pyx_v_itemsize == 0)) {
+ PyErr_SetString(PyExc_ZeroDivisionError, "integer division or modulo by zero");
+ __PYX_ERR(1, 180, __pyx_L1_error)
+ }
+ else if (sizeof(Py_ssize_t) == sizeof(long) && (!(((Py_ssize_t)-1) > 0)) && unlikely(__pyx_v_itemsize == (Py_ssize_t)-1) && unlikely(UNARY_NEG_WOULD_OVERFLOW(__pyx_v_self->len))) {
+ PyErr_SetString(PyExc_OverflowError, "value too large to perform division");
+ __PYX_ERR(1, 180, __pyx_L1_error)
+ }
+ __pyx_t_1 = __Pyx_div_Py_ssize_t(__pyx_v_self->len, __pyx_v_itemsize);
+ __pyx_t_9 = __pyx_t_1;
+ for (__pyx_t_11 = 0; __pyx_t_11 < __pyx_t_9; __pyx_t_11+=1) {
+ __pyx_v_i = __pyx_t_11;
+
+ /* "View.MemoryView":181
+ * p = self.data
+ * for i in range(self.len / itemsize):
+ * p[i] = Py_None # <<<<<<<<<<<<<<
+ * Py_INCREF(Py_None)
+ *
+ */
+ (__pyx_v_p[__pyx_v_i]) = Py_None;
+
+ /* "View.MemoryView":182
+ * for i in range(self.len / itemsize):
+ * p[i] = Py_None
+ * Py_INCREF(Py_None) # <<<<<<<<<<<<<<
+ *
+ * @cname('getbuffer')
+ */
+ Py_INCREF(Py_None);
+ }
+
+ /* "View.MemoryView":178
+ * raise MemoryError("unable to allocate array data.")
+ *
+ * if self.dtype_is_object: # <<<<<<<<<<<<<<
+ * p = self.data
+ * for i in range(self.len / itemsize):
+ */
+ }
+
+ /* "View.MemoryView":171
+ * self.free_data = allocate_buffer
+ * self.dtype_is_object = format == b'O'
+ * if allocate_buffer: # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ }
+
+ /* "View.MemoryView":122
+ * cdef bint dtype_is_object
+ *
+ * def __cinit__(array self, tuple shape, Py_ssize_t itemsize, format not None, # <<<<<<<<<<<<<<
+ * mode="c", bint allocate_buffer=True):
+ *
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_XDECREF(__pyx_t_6);
+ __Pyx_XDECREF(__pyx_t_10);
+ __Pyx_AddTraceback("View.MemoryView.array.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = -1;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_format);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":185
+ *
+ * @cname('getbuffer')
+ * def __getbuffer__(self, Py_buffer *info, int flags): # <<<<<<<<<<<<<<
+ * cdef int bufmode = -1
+ * if self.mode == u"c":
+ */
+
+/* Python wrapper */
+static CYTHON_UNUSED int __pyx_array_getbuffer(PyObject *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags); /*proto*/
+static CYTHON_UNUSED int __pyx_array_getbuffer(PyObject *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__getbuffer__ (wrapper)", 0);
+ __pyx_r = __pyx_array___pyx_pf_15View_dot_MemoryView_5array_2__getbuffer__(((struct __pyx_array_obj *)__pyx_v_self), ((Py_buffer *)__pyx_v_info), ((int)__pyx_v_flags));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_array___pyx_pf_15View_dot_MemoryView_5array_2__getbuffer__(struct __pyx_array_obj *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags) {
+ int __pyx_v_bufmode;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ char *__pyx_t_4;
+ Py_ssize_t __pyx_t_5;
+ int __pyx_t_6;
+ Py_ssize_t *__pyx_t_7;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ if (__pyx_v_info == NULL) {
+ PyErr_SetString(PyExc_BufferError, "PyObject_GetBuffer: view==NULL argument is obsolete");
+ return -1;
+ }
+ __Pyx_RefNannySetupContext("__getbuffer__", 0);
+ __pyx_v_info->obj = Py_None; __Pyx_INCREF(Py_None);
+ __Pyx_GIVEREF(__pyx_v_info->obj);
+
+ /* "View.MemoryView":186
+ * @cname('getbuffer')
+ * def __getbuffer__(self, Py_buffer *info, int flags):
+ * cdef int bufmode = -1 # <<<<<<<<<<<<<<
+ * if self.mode == u"c":
+ * bufmode = PyBUF_C_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ */
+ __pyx_v_bufmode = -1;
+
+ /* "View.MemoryView":187
+ * def __getbuffer__(self, Py_buffer *info, int flags):
+ * cdef int bufmode = -1
+ * if self.mode == u"c": # <<<<<<<<<<<<<<
+ * bufmode = PyBUF_C_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ * elif self.mode == u"fortran":
+ */
+ __pyx_t_1 = (__Pyx_PyUnicode_Equals(__pyx_v_self->mode, __pyx_n_u_c, Py_EQ)); if (unlikely(__pyx_t_1 < 0)) __PYX_ERR(1, 187, __pyx_L1_error)
+ __pyx_t_2 = (__pyx_t_1 != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":188
+ * cdef int bufmode = -1
+ * if self.mode == u"c":
+ * bufmode = PyBUF_C_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS # <<<<<<<<<<<<<<
+ * elif self.mode == u"fortran":
+ * bufmode = PyBUF_F_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ */
+ __pyx_v_bufmode = (PyBUF_C_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS);
+
+ /* "View.MemoryView":187
+ * def __getbuffer__(self, Py_buffer *info, int flags):
+ * cdef int bufmode = -1
+ * if self.mode == u"c": # <<<<<<<<<<<<<<
+ * bufmode = PyBUF_C_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ * elif self.mode == u"fortran":
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":189
+ * if self.mode == u"c":
+ * bufmode = PyBUF_C_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ * elif self.mode == u"fortran": # <<<<<<<<<<<<<<
+ * bufmode = PyBUF_F_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ * if not (flags & bufmode):
+ */
+ __pyx_t_2 = (__Pyx_PyUnicode_Equals(__pyx_v_self->mode, __pyx_n_u_fortran, Py_EQ)); if (unlikely(__pyx_t_2 < 0)) __PYX_ERR(1, 189, __pyx_L1_error)
+ __pyx_t_1 = (__pyx_t_2 != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":190
+ * bufmode = PyBUF_C_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ * elif self.mode == u"fortran":
+ * bufmode = PyBUF_F_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS # <<<<<<<<<<<<<<
+ * if not (flags & bufmode):
+ * raise ValueError("Can only create a buffer that is contiguous in memory.")
+ */
+ __pyx_v_bufmode = (PyBUF_F_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS);
+
+ /* "View.MemoryView":189
+ * if self.mode == u"c":
+ * bufmode = PyBUF_C_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ * elif self.mode == u"fortran": # <<<<<<<<<<<<<<
+ * bufmode = PyBUF_F_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ * if not (flags & bufmode):
+ */
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":191
+ * elif self.mode == u"fortran":
+ * bufmode = PyBUF_F_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ * if not (flags & bufmode): # <<<<<<<<<<<<<<
+ * raise ValueError("Can only create a buffer that is contiguous in memory.")
+ * info.buf = self.data
+ */
+ __pyx_t_1 = ((!((__pyx_v_flags & __pyx_v_bufmode) != 0)) != 0);
+ if (unlikely(__pyx_t_1)) {
+
+ /* "View.MemoryView":192
+ * bufmode = PyBUF_F_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ * if not (flags & bufmode):
+ * raise ValueError("Can only create a buffer that is contiguous in memory.") # <<<<<<<<<<<<<<
+ * info.buf = self.data
+ * info.len = self.len
+ */
+ __pyx_t_3 = __Pyx_PyObject_Call(__pyx_builtin_ValueError, __pyx_tuple__9, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 192, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_Raise(__pyx_t_3, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __PYX_ERR(1, 192, __pyx_L1_error)
+
+ /* "View.MemoryView":191
+ * elif self.mode == u"fortran":
+ * bufmode = PyBUF_F_CONTIGUOUS | PyBUF_ANY_CONTIGUOUS
+ * if not (flags & bufmode): # <<<<<<<<<<<<<<
+ * raise ValueError("Can only create a buffer that is contiguous in memory.")
+ * info.buf = self.data
+ */
+ }
+
+ /* "View.MemoryView":193
+ * if not (flags & bufmode):
+ * raise ValueError("Can only create a buffer that is contiguous in memory.")
+ * info.buf = self.data # <<<<<<<<<<<<<<
+ * info.len = self.len
+ * info.ndim = self.ndim
+ */
+ __pyx_t_4 = __pyx_v_self->data;
+ __pyx_v_info->buf = __pyx_t_4;
+
+ /* "View.MemoryView":194
+ * raise ValueError("Can only create a buffer that is contiguous in memory.")
+ * info.buf = self.data
+ * info.len = self.len # <<<<<<<<<<<<<<
+ * info.ndim = self.ndim
+ * info.shape = self._shape
+ */
+ __pyx_t_5 = __pyx_v_self->len;
+ __pyx_v_info->len = __pyx_t_5;
+
+ /* "View.MemoryView":195
+ * info.buf = self.data
+ * info.len = self.len
+ * info.ndim = self.ndim # <<<<<<<<<<<<<<
+ * info.shape = self._shape
+ * info.strides = self._strides
+ */
+ __pyx_t_6 = __pyx_v_self->ndim;
+ __pyx_v_info->ndim = __pyx_t_6;
+
+ /* "View.MemoryView":196
+ * info.len = self.len
+ * info.ndim = self.ndim
+ * info.shape = self._shape # <<<<<<<<<<<<<<
+ * info.strides = self._strides
+ * info.suboffsets = NULL
+ */
+ __pyx_t_7 = __pyx_v_self->_shape;
+ __pyx_v_info->shape = __pyx_t_7;
+
+ /* "View.MemoryView":197
+ * info.ndim = self.ndim
+ * info.shape = self._shape
+ * info.strides = self._strides # <<<<<<<<<<<<<<
+ * info.suboffsets = NULL
+ * info.itemsize = self.itemsize
+ */
+ __pyx_t_7 = __pyx_v_self->_strides;
+ __pyx_v_info->strides = __pyx_t_7;
+
+ /* "View.MemoryView":198
+ * info.shape = self._shape
+ * info.strides = self._strides
+ * info.suboffsets = NULL # <<<<<<<<<<<<<<
+ * info.itemsize = self.itemsize
+ * info.readonly = 0
+ */
+ __pyx_v_info->suboffsets = NULL;
+
+ /* "View.MemoryView":199
+ * info.strides = self._strides
+ * info.suboffsets = NULL
+ * info.itemsize = self.itemsize # <<<<<<<<<<<<<<
+ * info.readonly = 0
+ *
+ */
+ __pyx_t_5 = __pyx_v_self->itemsize;
+ __pyx_v_info->itemsize = __pyx_t_5;
+
+ /* "View.MemoryView":200
+ * info.suboffsets = NULL
+ * info.itemsize = self.itemsize
+ * info.readonly = 0 # <<<<<<<<<<<<<<
+ *
+ * if flags & PyBUF_FORMAT:
+ */
+ __pyx_v_info->readonly = 0;
+
+ /* "View.MemoryView":202
+ * info.readonly = 0
+ *
+ * if flags & PyBUF_FORMAT: # <<<<<<<<<<<<<<
+ * info.format = self.format
+ * else:
+ */
+ __pyx_t_1 = ((__pyx_v_flags & PyBUF_FORMAT) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":203
+ *
+ * if flags & PyBUF_FORMAT:
+ * info.format = self.format # <<<<<<<<<<<<<<
+ * else:
+ * info.format = NULL
+ */
+ __pyx_t_4 = __pyx_v_self->format;
+ __pyx_v_info->format = __pyx_t_4;
+
+ /* "View.MemoryView":202
+ * info.readonly = 0
+ *
+ * if flags & PyBUF_FORMAT: # <<<<<<<<<<<<<<
+ * info.format = self.format
+ * else:
+ */
+ goto __pyx_L5;
+ }
+
+ /* "View.MemoryView":205
+ * info.format = self.format
+ * else:
+ * info.format = NULL # <<<<<<<<<<<<<<
+ *
+ * info.obj = self
+ */
+ /*else*/ {
+ __pyx_v_info->format = NULL;
+ }
+ __pyx_L5:;
+
+ /* "View.MemoryView":207
+ * info.format = NULL
+ *
+ * info.obj = self # <<<<<<<<<<<<<<
+ *
+ * __pyx_getbuffer = capsule( &__pyx_array_getbuffer, "getbuffer(obj, view, flags)")
+ */
+ __Pyx_INCREF(((PyObject *)__pyx_v_self));
+ __Pyx_GIVEREF(((PyObject *)__pyx_v_self));
+ __Pyx_GOTREF(__pyx_v_info->obj);
+ __Pyx_DECREF(__pyx_v_info->obj);
+ __pyx_v_info->obj = ((PyObject *)__pyx_v_self);
+
+ /* "View.MemoryView":185
+ *
+ * @cname('getbuffer')
+ * def __getbuffer__(self, Py_buffer *info, int flags): # <<<<<<<<<<<<<<
+ * cdef int bufmode = -1
+ * if self.mode == u"c":
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.array.__getbuffer__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = -1;
+ if (__pyx_v_info->obj != NULL) {
+ __Pyx_GOTREF(__pyx_v_info->obj);
+ __Pyx_DECREF(__pyx_v_info->obj); __pyx_v_info->obj = 0;
+ }
+ goto __pyx_L2;
+ __pyx_L0:;
+ if (__pyx_v_info->obj == Py_None) {
+ __Pyx_GOTREF(__pyx_v_info->obj);
+ __Pyx_DECREF(__pyx_v_info->obj); __pyx_v_info->obj = 0;
+ }
+ __pyx_L2:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":211
+ * __pyx_getbuffer = capsule( &__pyx_array_getbuffer, "getbuffer(obj, view, flags)")
+ *
+ * def __dealloc__(array self): # <<<<<<<<<<<<<<
+ * if self.callback_free_data != NULL:
+ * self.callback_free_data(self.data)
+ */
+
+/* Python wrapper */
+static void __pyx_array___dealloc__(PyObject *__pyx_v_self); /*proto*/
+static void __pyx_array___dealloc__(PyObject *__pyx_v_self) {
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__dealloc__ (wrapper)", 0);
+ __pyx_array___pyx_pf_15View_dot_MemoryView_5array_4__dealloc__(((struct __pyx_array_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+}
+
+static void __pyx_array___pyx_pf_15View_dot_MemoryView_5array_4__dealloc__(struct __pyx_array_obj *__pyx_v_self) {
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ __Pyx_RefNannySetupContext("__dealloc__", 0);
+
+ /* "View.MemoryView":212
+ *
+ * def __dealloc__(array self):
+ * if self.callback_free_data != NULL: # <<<<<<<<<<<<<<
+ * self.callback_free_data(self.data)
+ * elif self.free_data:
+ */
+ __pyx_t_1 = ((__pyx_v_self->callback_free_data != NULL) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":213
+ * def __dealloc__(array self):
+ * if self.callback_free_data != NULL:
+ * self.callback_free_data(self.data) # <<<<<<<<<<<<<<
+ * elif self.free_data:
+ * if self.dtype_is_object:
+ */
+ __pyx_v_self->callback_free_data(__pyx_v_self->data);
+
+ /* "View.MemoryView":212
+ *
+ * def __dealloc__(array self):
+ * if self.callback_free_data != NULL: # <<<<<<<<<<<<<<
+ * self.callback_free_data(self.data)
+ * elif self.free_data:
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":214
+ * if self.callback_free_data != NULL:
+ * self.callback_free_data(self.data)
+ * elif self.free_data: # <<<<<<<<<<<<<<
+ * if self.dtype_is_object:
+ * refcount_objects_in_slice(self.data, self._shape,
+ */
+ __pyx_t_1 = (__pyx_v_self->free_data != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":215
+ * self.callback_free_data(self.data)
+ * elif self.free_data:
+ * if self.dtype_is_object: # <<<<<<<<<<<<<<
+ * refcount_objects_in_slice(self.data, self._shape,
+ * self._strides, self.ndim, False)
+ */
+ __pyx_t_1 = (__pyx_v_self->dtype_is_object != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":216
+ * elif self.free_data:
+ * if self.dtype_is_object:
+ * refcount_objects_in_slice(self.data, self._shape, # <<<<<<<<<<<<<<
+ * self._strides, self.ndim, False)
+ * free(self.data)
+ */
+ __pyx_memoryview_refcount_objects_in_slice(__pyx_v_self->data, __pyx_v_self->_shape, __pyx_v_self->_strides, __pyx_v_self->ndim, 0);
+
+ /* "View.MemoryView":215
+ * self.callback_free_data(self.data)
+ * elif self.free_data:
+ * if self.dtype_is_object: # <<<<<<<<<<<<<<
+ * refcount_objects_in_slice(self.data, self._shape,
+ * self._strides, self.ndim, False)
+ */
+ }
+
+ /* "View.MemoryView":218
+ * refcount_objects_in_slice(self.data, self._shape,
+ * self._strides, self.ndim, False)
+ * free(self.data) # <<<<<<<<<<<<<<
+ * PyObject_Free(self._shape)
+ *
+ */
+ free(__pyx_v_self->data);
+
+ /* "View.MemoryView":214
+ * if self.callback_free_data != NULL:
+ * self.callback_free_data(self.data)
+ * elif self.free_data: # <<<<<<<<<<<<<<
+ * if self.dtype_is_object:
+ * refcount_objects_in_slice(self.data, self._shape,
+ */
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":219
+ * self._strides, self.ndim, False)
+ * free(self.data)
+ * PyObject_Free(self._shape) # <<<<<<<<<<<<<<
+ *
+ * @property
+ */
+ PyObject_Free(__pyx_v_self->_shape);
+
+ /* "View.MemoryView":211
+ * __pyx_getbuffer = capsule( &__pyx_array_getbuffer, "getbuffer(obj, view, flags)")
+ *
+ * def __dealloc__(array self): # <<<<<<<<<<<<<<
+ * if self.callback_free_data != NULL:
+ * self.callback_free_data(self.data)
+ */
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+}
+
+/* "View.MemoryView":222
+ *
+ * @property
+ * def memview(self): # <<<<<<<<<<<<<<
+ * return self.get_memview()
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_5array_7memview_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_5array_7memview_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_5array_7memview___get__(((struct __pyx_array_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_5array_7memview___get__(struct __pyx_array_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":223
+ * @property
+ * def memview(self):
+ * return self.get_memview() # <<<<<<<<<<<<<<
+ *
+ * @cname('get_memview')
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = ((struct __pyx_vtabstruct_array *)__pyx_v_self->__pyx_vtab)->get_memview(__pyx_v_self); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 223, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":222
+ *
+ * @property
+ * def memview(self): # <<<<<<<<<<<<<<
+ * return self.get_memview()
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.array.memview.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":226
+ *
+ * @cname('get_memview')
+ * cdef get_memview(self): # <<<<<<<<<<<<<<
+ * flags = PyBUF_ANY_CONTIGUOUS|PyBUF_FORMAT|PyBUF_WRITABLE
+ * return memoryview(self, flags, self.dtype_is_object)
+ */
+
+static PyObject *__pyx_array_get_memview(struct __pyx_array_obj *__pyx_v_self) {
+ int __pyx_v_flags;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("get_memview", 0);
+
+ /* "View.MemoryView":227
+ * @cname('get_memview')
+ * cdef get_memview(self):
+ * flags = PyBUF_ANY_CONTIGUOUS|PyBUF_FORMAT|PyBUF_WRITABLE # <<<<<<<<<<<<<<
+ * return memoryview(self, flags, self.dtype_is_object)
+ *
+ */
+ __pyx_v_flags = ((PyBUF_ANY_CONTIGUOUS | PyBUF_FORMAT) | PyBUF_WRITABLE);
+
+ /* "View.MemoryView":228
+ * cdef get_memview(self):
+ * flags = PyBUF_ANY_CONTIGUOUS|PyBUF_FORMAT|PyBUF_WRITABLE
+ * return memoryview(self, flags, self.dtype_is_object) # <<<<<<<<<<<<<<
+ *
+ * def __len__(self):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_flags); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 228, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_2 = __Pyx_PyBool_FromLong(__pyx_v_self->dtype_is_object); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 228, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_3 = PyTuple_New(3); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 228, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_INCREF(((PyObject *)__pyx_v_self));
+ __Pyx_GIVEREF(((PyObject *)__pyx_v_self));
+ PyTuple_SET_ITEM(__pyx_t_3, 0, ((PyObject *)__pyx_v_self));
+ __Pyx_GIVEREF(__pyx_t_1);
+ PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_t_1);
+ __Pyx_GIVEREF(__pyx_t_2);
+ PyTuple_SET_ITEM(__pyx_t_3, 2, __pyx_t_2);
+ __pyx_t_1 = 0;
+ __pyx_t_2 = 0;
+ __pyx_t_2 = __Pyx_PyObject_Call(((PyObject *)__pyx_memoryview_type), __pyx_t_3, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 228, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":226
+ *
+ * @cname('get_memview')
+ * cdef get_memview(self): # <<<<<<<<<<<<<<
+ * flags = PyBUF_ANY_CONTIGUOUS|PyBUF_FORMAT|PyBUF_WRITABLE
+ * return memoryview(self, flags, self.dtype_is_object)
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.array.get_memview", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":230
+ * return memoryview(self, flags, self.dtype_is_object)
+ *
+ * def __len__(self): # <<<<<<<<<<<<<<
+ * return self._shape[0]
+ *
+ */
+
+/* Python wrapper */
+static Py_ssize_t __pyx_array___len__(PyObject *__pyx_v_self); /*proto*/
+static Py_ssize_t __pyx_array___len__(PyObject *__pyx_v_self) {
+ Py_ssize_t __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__len__ (wrapper)", 0);
+ __pyx_r = __pyx_array___pyx_pf_15View_dot_MemoryView_5array_6__len__(((struct __pyx_array_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static Py_ssize_t __pyx_array___pyx_pf_15View_dot_MemoryView_5array_6__len__(struct __pyx_array_obj *__pyx_v_self) {
+ Py_ssize_t __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__len__", 0);
+
+ /* "View.MemoryView":231
+ *
+ * def __len__(self):
+ * return self._shape[0] # <<<<<<<<<<<<<<
+ *
+ * def __getattr__(self, attr):
+ */
+ __pyx_r = (__pyx_v_self->_shape[0]);
+ goto __pyx_L0;
+
+ /* "View.MemoryView":230
+ * return memoryview(self, flags, self.dtype_is_object)
+ *
+ * def __len__(self): # <<<<<<<<<<<<<<
+ * return self._shape[0]
+ *
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":233
+ * return self._shape[0]
+ *
+ * def __getattr__(self, attr): # <<<<<<<<<<<<<<
+ * return getattr(self.memview, attr)
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_array___getattr__(PyObject *__pyx_v_self, PyObject *__pyx_v_attr); /*proto*/
+static PyObject *__pyx_array___getattr__(PyObject *__pyx_v_self, PyObject *__pyx_v_attr) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__getattr__ (wrapper)", 0);
+ __pyx_r = __pyx_array___pyx_pf_15View_dot_MemoryView_5array_8__getattr__(((struct __pyx_array_obj *)__pyx_v_self), ((PyObject *)__pyx_v_attr));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_array___pyx_pf_15View_dot_MemoryView_5array_8__getattr__(struct __pyx_array_obj *__pyx_v_self, PyObject *__pyx_v_attr) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__getattr__", 0);
+
+ /* "View.MemoryView":234
+ *
+ * def __getattr__(self, attr):
+ * return getattr(self.memview, attr) # <<<<<<<<<<<<<<
+ *
+ * def __getitem__(self, item):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_self), __pyx_n_s_memview); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 234, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_2 = __Pyx_GetAttr(__pyx_t_1, __pyx_v_attr); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 234, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":233
+ * return self._shape[0]
+ *
+ * def __getattr__(self, attr): # <<<<<<<<<<<<<<
+ * return getattr(self.memview, attr)
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_AddTraceback("View.MemoryView.array.__getattr__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":236
+ * return getattr(self.memview, attr)
+ *
+ * def __getitem__(self, item): # <<<<<<<<<<<<<<
+ * return self.memview[item]
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_array___getitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_item); /*proto*/
+static PyObject *__pyx_array___getitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_item) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__getitem__ (wrapper)", 0);
+ __pyx_r = __pyx_array___pyx_pf_15View_dot_MemoryView_5array_10__getitem__(((struct __pyx_array_obj *)__pyx_v_self), ((PyObject *)__pyx_v_item));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_array___pyx_pf_15View_dot_MemoryView_5array_10__getitem__(struct __pyx_array_obj *__pyx_v_self, PyObject *__pyx_v_item) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__getitem__", 0);
+
+ /* "View.MemoryView":237
+ *
+ * def __getitem__(self, item):
+ * return self.memview[item] # <<<<<<<<<<<<<<
+ *
+ * def __setitem__(self, item, value):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_self), __pyx_n_s_memview); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 237, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_2 = __Pyx_PyObject_GetItem(__pyx_t_1, __pyx_v_item); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 237, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":236
+ * return getattr(self.memview, attr)
+ *
+ * def __getitem__(self, item): # <<<<<<<<<<<<<<
+ * return self.memview[item]
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_AddTraceback("View.MemoryView.array.__getitem__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":239
+ * return self.memview[item]
+ *
+ * def __setitem__(self, item, value): # <<<<<<<<<<<<<<
+ * self.memview[item] = value
+ *
+ */
+
+/* Python wrapper */
+static int __pyx_array___setitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_item, PyObject *__pyx_v_value); /*proto*/
+static int __pyx_array___setitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_item, PyObject *__pyx_v_value) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__setitem__ (wrapper)", 0);
+ __pyx_r = __pyx_array___pyx_pf_15View_dot_MemoryView_5array_12__setitem__(((struct __pyx_array_obj *)__pyx_v_self), ((PyObject *)__pyx_v_item), ((PyObject *)__pyx_v_value));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_array___pyx_pf_15View_dot_MemoryView_5array_12__setitem__(struct __pyx_array_obj *__pyx_v_self, PyObject *__pyx_v_item, PyObject *__pyx_v_value) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__setitem__", 0);
+
+ /* "View.MemoryView":240
+ *
+ * def __setitem__(self, item, value):
+ * self.memview[item] = value # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_self), __pyx_n_s_memview); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 240, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ if (unlikely(PyObject_SetItem(__pyx_t_1, __pyx_v_item, __pyx_v_value) < 0)) __PYX_ERR(1, 240, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+
+ /* "View.MemoryView":239
+ * return self.memview[item]
+ *
+ * def __setitem__(self, item, value): # <<<<<<<<<<<<<<
+ * self.memview[item] = value
+ *
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.array.__setitem__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = -1;
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "(tree fragment)":1
+ * def __reduce_cython__(self): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw___pyx_array_1__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/
+static PyObject *__pyx_pw___pyx_array_1__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0);
+ __pyx_r = __pyx_pf___pyx_array___reduce_cython__(((struct __pyx_array_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf___pyx_array___reduce_cython__(CYTHON_UNUSED struct __pyx_array_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__reduce_cython__", 0);
+
+ /* "(tree fragment)":2
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<<
+ * def __setstate_cython__(self, __pyx_state):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__10, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 2, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_Raise(__pyx_t_1, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __PYX_ERR(1, 2, __pyx_L1_error)
+
+ /* "(tree fragment)":1
+ * def __reduce_cython__(self): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.array.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "(tree fragment)":3
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw___pyx_array_3__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/
+static PyObject *__pyx_pw___pyx_array_3__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0);
+ __pyx_r = __pyx_pf___pyx_array_2__setstate_cython__(((struct __pyx_array_obj *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf___pyx_array_2__setstate_cython__(CYTHON_UNUSED struct __pyx_array_obj *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__setstate_cython__", 0);
+
+ /* "(tree fragment)":4
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<<
+ */
+ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__11, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_Raise(__pyx_t_1, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __PYX_ERR(1, 4, __pyx_L1_error)
+
+ /* "(tree fragment)":3
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.array.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":244
+ *
+ * @cname("__pyx_array_new")
+ * cdef array array_cwrapper(tuple shape, Py_ssize_t itemsize, char *format, # <<<<<<<<<<<<<<
+ * char *mode, char *buf):
+ * cdef array result
+ */
+
+static struct __pyx_array_obj *__pyx_array_new(PyObject *__pyx_v_shape, Py_ssize_t __pyx_v_itemsize, char *__pyx_v_format, char *__pyx_v_mode, char *__pyx_v_buf) {
+ struct __pyx_array_obj *__pyx_v_result = 0;
+ struct __pyx_array_obj *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ PyObject *__pyx_t_4 = NULL;
+ PyObject *__pyx_t_5 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("array_cwrapper", 0);
+
+ /* "View.MemoryView":248
+ * cdef array result
+ *
+ * if buf == NULL: # <<<<<<<<<<<<<<
+ * result = array(shape, itemsize, format, mode.decode('ASCII'))
+ * else:
+ */
+ __pyx_t_1 = ((__pyx_v_buf == NULL) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":249
+ *
+ * if buf == NULL:
+ * result = array(shape, itemsize, format, mode.decode('ASCII')) # <<<<<<<<<<<<<<
+ * else:
+ * result = array(shape, itemsize, format, mode.decode('ASCII'),
+ */
+ __pyx_t_2 = PyInt_FromSsize_t(__pyx_v_itemsize); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 249, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_3 = __Pyx_PyBytes_FromString(__pyx_v_format); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 249, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_t_4 = __Pyx_decode_c_string(__pyx_v_mode, 0, strlen(__pyx_v_mode), NULL, NULL, PyUnicode_DecodeASCII); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 249, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_5 = PyTuple_New(4); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 249, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_INCREF(__pyx_v_shape);
+ __Pyx_GIVEREF(__pyx_v_shape);
+ PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_v_shape);
+ __Pyx_GIVEREF(__pyx_t_2);
+ PyTuple_SET_ITEM(__pyx_t_5, 1, __pyx_t_2);
+ __Pyx_GIVEREF(__pyx_t_3);
+ PyTuple_SET_ITEM(__pyx_t_5, 2, __pyx_t_3);
+ __Pyx_GIVEREF(__pyx_t_4);
+ PyTuple_SET_ITEM(__pyx_t_5, 3, __pyx_t_4);
+ __pyx_t_2 = 0;
+ __pyx_t_3 = 0;
+ __pyx_t_4 = 0;
+ __pyx_t_4 = __Pyx_PyObject_Call(((PyObject *)__pyx_array_type), __pyx_t_5, NULL); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 249, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
+ __pyx_v_result = ((struct __pyx_array_obj *)__pyx_t_4);
+ __pyx_t_4 = 0;
+
+ /* "View.MemoryView":248
+ * cdef array result
+ *
+ * if buf == NULL: # <<<<<<<<<<<<<<
+ * result = array(shape, itemsize, format, mode.decode('ASCII'))
+ * else:
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":251
+ * result = array(shape, itemsize, format, mode.decode('ASCII'))
+ * else:
+ * result = array(shape, itemsize, format, mode.decode('ASCII'), # <<<<<<<<<<<<<<
+ * allocate_buffer=False)
+ * result.data = buf
+ */
+ /*else*/ {
+ __pyx_t_4 = PyInt_FromSsize_t(__pyx_v_itemsize); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 251, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_5 = __Pyx_PyBytes_FromString(__pyx_v_format); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 251, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __pyx_t_3 = __Pyx_decode_c_string(__pyx_v_mode, 0, strlen(__pyx_v_mode), NULL, NULL, PyUnicode_DecodeASCII); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 251, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_t_2 = PyTuple_New(4); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 251, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_INCREF(__pyx_v_shape);
+ __Pyx_GIVEREF(__pyx_v_shape);
+ PyTuple_SET_ITEM(__pyx_t_2, 0, __pyx_v_shape);
+ __Pyx_GIVEREF(__pyx_t_4);
+ PyTuple_SET_ITEM(__pyx_t_2, 1, __pyx_t_4);
+ __Pyx_GIVEREF(__pyx_t_5);
+ PyTuple_SET_ITEM(__pyx_t_2, 2, __pyx_t_5);
+ __Pyx_GIVEREF(__pyx_t_3);
+ PyTuple_SET_ITEM(__pyx_t_2, 3, __pyx_t_3);
+ __pyx_t_4 = 0;
+ __pyx_t_5 = 0;
+ __pyx_t_3 = 0;
+
+ /* "View.MemoryView":252
+ * else:
+ * result = array(shape, itemsize, format, mode.decode('ASCII'),
+ * allocate_buffer=False) # <<<<<<<<<<<<<<
+ * result.data = buf
+ *
+ */
+ __pyx_t_3 = __Pyx_PyDict_NewPresized(1); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 252, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ if (PyDict_SetItem(__pyx_t_3, __pyx_n_s_allocate_buffer, Py_False) < 0) __PYX_ERR(1, 252, __pyx_L1_error)
+
+ /* "View.MemoryView":251
+ * result = array(shape, itemsize, format, mode.decode('ASCII'))
+ * else:
+ * result = array(shape, itemsize, format, mode.decode('ASCII'), # <<<<<<<<<<<<<<
+ * allocate_buffer=False)
+ * result.data = buf
+ */
+ __pyx_t_5 = __Pyx_PyObject_Call(((PyObject *)__pyx_array_type), __pyx_t_2, __pyx_t_3); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 251, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __pyx_v_result = ((struct __pyx_array_obj *)__pyx_t_5);
+ __pyx_t_5 = 0;
+
+ /* "View.MemoryView":253
+ * result = array(shape, itemsize, format, mode.decode('ASCII'),
+ * allocate_buffer=False)
+ * result.data = buf # <<<<<<<<<<<<<<
+ *
+ * return result
+ */
+ __pyx_v_result->data = __pyx_v_buf;
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":255
+ * result.data = buf
+ *
+ * return result # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __Pyx_XDECREF(((PyObject *)__pyx_r));
+ __Pyx_INCREF(((PyObject *)__pyx_v_result));
+ __pyx_r = __pyx_v_result;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":244
+ *
+ * @cname("__pyx_array_new")
+ * cdef array array_cwrapper(tuple shape, Py_ssize_t itemsize, char *format, # <<<<<<<<<<<<<<
+ * char *mode, char *buf):
+ * cdef array result
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_AddTraceback("View.MemoryView.array_cwrapper", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XDECREF((PyObject *)__pyx_v_result);
+ __Pyx_XGIVEREF((PyObject *)__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":281
+ * cdef class Enum(object):
+ * cdef object name
+ * def __init__(self, name): # <<<<<<<<<<<<<<
+ * self.name = name
+ * def __repr__(self):
+ */
+
+/* Python wrapper */
+static int __pyx_MemviewEnum___init__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
+static int __pyx_MemviewEnum___init__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
+ PyObject *__pyx_v_name = 0;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__init__ (wrapper)", 0);
+ {
+ static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_name,0};
+ PyObject* values[1] = {0};
+ if (unlikely(__pyx_kwds)) {
+ Py_ssize_t kw_args;
+ const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
+ switch (pos_args) {
+ case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ CYTHON_FALLTHROUGH;
+ case 0: break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ kw_args = PyDict_Size(__pyx_kwds);
+ switch (pos_args) {
+ case 0:
+ if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_name)) != 0)) kw_args--;
+ else goto __pyx_L5_argtuple_error;
+ }
+ if (unlikely(kw_args > 0)) {
+ if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "__init__") < 0)) __PYX_ERR(1, 281, __pyx_L3_error)
+ }
+ } else if (PyTuple_GET_SIZE(__pyx_args) != 1) {
+ goto __pyx_L5_argtuple_error;
+ } else {
+ values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ }
+ __pyx_v_name = values[0];
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L5_argtuple_error:;
+ __Pyx_RaiseArgtupleInvalid("__init__", 1, 1, 1, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(1, 281, __pyx_L3_error)
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("View.MemoryView.Enum.__init__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return -1;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_MemviewEnum___pyx_pf_15View_dot_MemoryView_4Enum___init__(((struct __pyx_MemviewEnum_obj *)__pyx_v_self), __pyx_v_name);
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_MemviewEnum___pyx_pf_15View_dot_MemoryView_4Enum___init__(struct __pyx_MemviewEnum_obj *__pyx_v_self, PyObject *__pyx_v_name) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__init__", 0);
+
+ /* "View.MemoryView":282
+ * cdef object name
+ * def __init__(self, name):
+ * self.name = name # <<<<<<<<<<<<<<
+ * def __repr__(self):
+ * return self.name
+ */
+ __Pyx_INCREF(__pyx_v_name);
+ __Pyx_GIVEREF(__pyx_v_name);
+ __Pyx_GOTREF(__pyx_v_self->name);
+ __Pyx_DECREF(__pyx_v_self->name);
+ __pyx_v_self->name = __pyx_v_name;
+
+ /* "View.MemoryView":281
+ * cdef class Enum(object):
+ * cdef object name
+ * def __init__(self, name): # <<<<<<<<<<<<<<
+ * self.name = name
+ * def __repr__(self):
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":283
+ * def __init__(self, name):
+ * self.name = name
+ * def __repr__(self): # <<<<<<<<<<<<<<
+ * return self.name
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_MemviewEnum___repr__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_MemviewEnum___repr__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__repr__ (wrapper)", 0);
+ __pyx_r = __pyx_MemviewEnum___pyx_pf_15View_dot_MemoryView_4Enum_2__repr__(((struct __pyx_MemviewEnum_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_MemviewEnum___pyx_pf_15View_dot_MemoryView_4Enum_2__repr__(struct __pyx_MemviewEnum_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__repr__", 0);
+
+ /* "View.MemoryView":284
+ * self.name = name
+ * def __repr__(self):
+ * return self.name # <<<<<<<<<<<<<<
+ *
+ * cdef generic = Enum("")
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(__pyx_v_self->name);
+ __pyx_r = __pyx_v_self->name;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":283
+ * def __init__(self, name):
+ * self.name = name
+ * def __repr__(self): # <<<<<<<<<<<<<<
+ * return self.name
+ *
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "(tree fragment)":1
+ * def __reduce_cython__(self): # <<<<<<<<<<<<<<
+ * cdef tuple state
+ * cdef object _dict
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw___pyx_MemviewEnum_1__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/
+static PyObject *__pyx_pw___pyx_MemviewEnum_1__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0);
+ __pyx_r = __pyx_pf___pyx_MemviewEnum___reduce_cython__(((struct __pyx_MemviewEnum_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf___pyx_MemviewEnum___reduce_cython__(struct __pyx_MemviewEnum_obj *__pyx_v_self) {
+ PyObject *__pyx_v_state = 0;
+ PyObject *__pyx_v__dict = 0;
+ int __pyx_v_use_setstate;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_t_2;
+ int __pyx_t_3;
+ PyObject *__pyx_t_4 = NULL;
+ PyObject *__pyx_t_5 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__reduce_cython__", 0);
+
+ /* "(tree fragment)":5
+ * cdef object _dict
+ * cdef bint use_setstate
+ * state = (self.name,) # <<<<<<<<<<<<<<
+ * _dict = getattr(self, '__dict__', None)
+ * if _dict is not None:
+ */
+ __pyx_t_1 = PyTuple_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 5, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_INCREF(__pyx_v_self->name);
+ __Pyx_GIVEREF(__pyx_v_self->name);
+ PyTuple_SET_ITEM(__pyx_t_1, 0, __pyx_v_self->name);
+ __pyx_v_state = ((PyObject*)__pyx_t_1);
+ __pyx_t_1 = 0;
+
+ /* "(tree fragment)":6
+ * cdef bint use_setstate
+ * state = (self.name,)
+ * _dict = getattr(self, '__dict__', None) # <<<<<<<<<<<<<<
+ * if _dict is not None:
+ * state += (_dict,)
+ */
+ __pyx_t_1 = __Pyx_GetAttr3(((PyObject *)__pyx_v_self), __pyx_n_s_dict, Py_None); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 6, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_v__dict = __pyx_t_1;
+ __pyx_t_1 = 0;
+
+ /* "(tree fragment)":7
+ * state = (self.name,)
+ * _dict = getattr(self, '__dict__', None)
+ * if _dict is not None: # <<<<<<<<<<<<<<
+ * state += (_dict,)
+ * use_setstate = True
+ */
+ __pyx_t_2 = (__pyx_v__dict != Py_None);
+ __pyx_t_3 = (__pyx_t_2 != 0);
+ if (__pyx_t_3) {
+
+ /* "(tree fragment)":8
+ * _dict = getattr(self, '__dict__', None)
+ * if _dict is not None:
+ * state += (_dict,) # <<<<<<<<<<<<<<
+ * use_setstate = True
+ * else:
+ */
+ __pyx_t_1 = PyTuple_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 8, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_INCREF(__pyx_v__dict);
+ __Pyx_GIVEREF(__pyx_v__dict);
+ PyTuple_SET_ITEM(__pyx_t_1, 0, __pyx_v__dict);
+ __pyx_t_4 = PyNumber_InPlaceAdd(__pyx_v_state, __pyx_t_1); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 8, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __Pyx_DECREF_SET(__pyx_v_state, ((PyObject*)__pyx_t_4));
+ __pyx_t_4 = 0;
+
+ /* "(tree fragment)":9
+ * if _dict is not None:
+ * state += (_dict,)
+ * use_setstate = True # <<<<<<<<<<<<<<
+ * else:
+ * use_setstate = self.name is not None
+ */
+ __pyx_v_use_setstate = 1;
+
+ /* "(tree fragment)":7
+ * state = (self.name,)
+ * _dict = getattr(self, '__dict__', None)
+ * if _dict is not None: # <<<<<<<<<<<<<<
+ * state += (_dict,)
+ * use_setstate = True
+ */
+ goto __pyx_L3;
+ }
+
+ /* "(tree fragment)":11
+ * use_setstate = True
+ * else:
+ * use_setstate = self.name is not None # <<<<<<<<<<<<<<
+ * if use_setstate:
+ * return __pyx_unpickle_Enum, (type(self), 0xb068931, None), state
+ */
+ /*else*/ {
+ __pyx_t_3 = (__pyx_v_self->name != Py_None);
+ __pyx_v_use_setstate = __pyx_t_3;
+ }
+ __pyx_L3:;
+
+ /* "(tree fragment)":12
+ * else:
+ * use_setstate = self.name is not None
+ * if use_setstate: # <<<<<<<<<<<<<<
+ * return __pyx_unpickle_Enum, (type(self), 0xb068931, None), state
+ * else:
+ */
+ __pyx_t_3 = (__pyx_v_use_setstate != 0);
+ if (__pyx_t_3) {
+
+ /* "(tree fragment)":13
+ * use_setstate = self.name is not None
+ * if use_setstate:
+ * return __pyx_unpickle_Enum, (type(self), 0xb068931, None), state # <<<<<<<<<<<<<<
+ * else:
+ * return __pyx_unpickle_Enum, (type(self), 0xb068931, state)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_GetModuleGlobalName(__pyx_t_4, __pyx_n_s_pyx_unpickle_Enum); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 13, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_1 = PyTuple_New(3); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 13, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_INCREF(((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self))));
+ __Pyx_GIVEREF(((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self))));
+ PyTuple_SET_ITEM(__pyx_t_1, 0, ((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self))));
+ __Pyx_INCREF(__pyx_int_184977713);
+ __Pyx_GIVEREF(__pyx_int_184977713);
+ PyTuple_SET_ITEM(__pyx_t_1, 1, __pyx_int_184977713);
+ __Pyx_INCREF(Py_None);
+ __Pyx_GIVEREF(Py_None);
+ PyTuple_SET_ITEM(__pyx_t_1, 2, Py_None);
+ __pyx_t_5 = PyTuple_New(3); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 13, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_GIVEREF(__pyx_t_4);
+ PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_t_4);
+ __Pyx_GIVEREF(__pyx_t_1);
+ PyTuple_SET_ITEM(__pyx_t_5, 1, __pyx_t_1);
+ __Pyx_INCREF(__pyx_v_state);
+ __Pyx_GIVEREF(__pyx_v_state);
+ PyTuple_SET_ITEM(__pyx_t_5, 2, __pyx_v_state);
+ __pyx_t_4 = 0;
+ __pyx_t_1 = 0;
+ __pyx_r = __pyx_t_5;
+ __pyx_t_5 = 0;
+ goto __pyx_L0;
+
+ /* "(tree fragment)":12
+ * else:
+ * use_setstate = self.name is not None
+ * if use_setstate: # <<<<<<<<<<<<<<
+ * return __pyx_unpickle_Enum, (type(self), 0xb068931, None), state
+ * else:
+ */
+ }
+
+ /* "(tree fragment)":15
+ * return __pyx_unpickle_Enum, (type(self), 0xb068931, None), state
+ * else:
+ * return __pyx_unpickle_Enum, (type(self), 0xb068931, state) # <<<<<<<<<<<<<<
+ * def __setstate_cython__(self, __pyx_state):
+ * __pyx_unpickle_Enum__set_state(self, __pyx_state)
+ */
+ /*else*/ {
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_GetModuleGlobalName(__pyx_t_5, __pyx_n_s_pyx_unpickle_Enum); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 15, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __pyx_t_1 = PyTuple_New(3); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 15, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_INCREF(((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self))));
+ __Pyx_GIVEREF(((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self))));
+ PyTuple_SET_ITEM(__pyx_t_1, 0, ((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self))));
+ __Pyx_INCREF(__pyx_int_184977713);
+ __Pyx_GIVEREF(__pyx_int_184977713);
+ PyTuple_SET_ITEM(__pyx_t_1, 1, __pyx_int_184977713);
+ __Pyx_INCREF(__pyx_v_state);
+ __Pyx_GIVEREF(__pyx_v_state);
+ PyTuple_SET_ITEM(__pyx_t_1, 2, __pyx_v_state);
+ __pyx_t_4 = PyTuple_New(2); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 15, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_GIVEREF(__pyx_t_5);
+ PyTuple_SET_ITEM(__pyx_t_4, 0, __pyx_t_5);
+ __Pyx_GIVEREF(__pyx_t_1);
+ PyTuple_SET_ITEM(__pyx_t_4, 1, __pyx_t_1);
+ __pyx_t_5 = 0;
+ __pyx_t_1 = 0;
+ __pyx_r = __pyx_t_4;
+ __pyx_t_4 = 0;
+ goto __pyx_L0;
+ }
+
+ /* "(tree fragment)":1
+ * def __reduce_cython__(self): # <<<<<<<<<<<<<<
+ * cdef tuple state
+ * cdef object _dict
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_AddTraceback("View.MemoryView.Enum.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_state);
+ __Pyx_XDECREF(__pyx_v__dict);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "(tree fragment)":16
+ * else:
+ * return __pyx_unpickle_Enum, (type(self), 0xb068931, state)
+ * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<<
+ * __pyx_unpickle_Enum__set_state(self, __pyx_state)
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw___pyx_MemviewEnum_3__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/
+static PyObject *__pyx_pw___pyx_MemviewEnum_3__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0);
+ __pyx_r = __pyx_pf___pyx_MemviewEnum_2__setstate_cython__(((struct __pyx_MemviewEnum_obj *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf___pyx_MemviewEnum_2__setstate_cython__(struct __pyx_MemviewEnum_obj *__pyx_v_self, PyObject *__pyx_v___pyx_state) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__setstate_cython__", 0);
+
+ /* "(tree fragment)":17
+ * return __pyx_unpickle_Enum, (type(self), 0xb068931, state)
+ * def __setstate_cython__(self, __pyx_state):
+ * __pyx_unpickle_Enum__set_state(self, __pyx_state) # <<<<<<<<<<<<<<
+ */
+ if (!(likely(PyTuple_CheckExact(__pyx_v___pyx_state))||((__pyx_v___pyx_state) == Py_None)||(PyErr_Format(PyExc_TypeError, "Expected %.16s, got %.200s", "tuple", Py_TYPE(__pyx_v___pyx_state)->tp_name), 0))) __PYX_ERR(1, 17, __pyx_L1_error)
+ __pyx_t_1 = __pyx_unpickle_Enum__set_state(__pyx_v_self, ((PyObject*)__pyx_v___pyx_state)); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 17, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+
+ /* "(tree fragment)":16
+ * else:
+ * return __pyx_unpickle_Enum, (type(self), 0xb068931, state)
+ * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<<
+ * __pyx_unpickle_Enum__set_state(self, __pyx_state)
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.Enum.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":298
+ *
+ * @cname('__pyx_align_pointer')
+ * cdef void *align_pointer(void *memory, size_t alignment) nogil: # <<<<<<<<<<<<<<
+ * "Align pointer memory on a given boundary"
+ * cdef Py_intptr_t aligned_p = memory
+ */
+
+static void *__pyx_align_pointer(void *__pyx_v_memory, size_t __pyx_v_alignment) {
+ Py_intptr_t __pyx_v_aligned_p;
+ size_t __pyx_v_offset;
+ void *__pyx_r;
+ int __pyx_t_1;
+
+ /* "View.MemoryView":300
+ * cdef void *align_pointer(void *memory, size_t alignment) nogil:
+ * "Align pointer memory on a given boundary"
+ * cdef Py_intptr_t aligned_p = memory # <<<<<<<<<<<<<<
+ * cdef size_t offset
+ *
+ */
+ __pyx_v_aligned_p = ((Py_intptr_t)__pyx_v_memory);
+
+ /* "View.MemoryView":304
+ *
+ * with cython.cdivision(True):
+ * offset = aligned_p % alignment # <<<<<<<<<<<<<<
+ *
+ * if offset > 0:
+ */
+ __pyx_v_offset = (__pyx_v_aligned_p % __pyx_v_alignment);
+
+ /* "View.MemoryView":306
+ * offset = aligned_p % alignment
+ *
+ * if offset > 0: # <<<<<<<<<<<<<<
+ * aligned_p += alignment - offset
+ *
+ */
+ __pyx_t_1 = ((__pyx_v_offset > 0) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":307
+ *
+ * if offset > 0:
+ * aligned_p += alignment - offset # <<<<<<<<<<<<<<
+ *
+ * return aligned_p
+ */
+ __pyx_v_aligned_p = (__pyx_v_aligned_p + (__pyx_v_alignment - __pyx_v_offset));
+
+ /* "View.MemoryView":306
+ * offset = aligned_p % alignment
+ *
+ * if offset > 0: # <<<<<<<<<<<<<<
+ * aligned_p += alignment - offset
+ *
+ */
+ }
+
+ /* "View.MemoryView":309
+ * aligned_p += alignment - offset
+ *
+ * return aligned_p # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_r = ((void *)__pyx_v_aligned_p);
+ goto __pyx_L0;
+
+ /* "View.MemoryView":298
+ *
+ * @cname('__pyx_align_pointer')
+ * cdef void *align_pointer(void *memory, size_t alignment) nogil: # <<<<<<<<<<<<<<
+ * "Align pointer memory on a given boundary"
+ * cdef Py_intptr_t aligned_p = memory
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ return __pyx_r;
+}
+
+/* "View.MemoryView":345
+ * cdef __Pyx_TypeInfo *typeinfo
+ *
+ * def __cinit__(memoryview self, object obj, int flags, bint dtype_is_object=False): # <<<<<<<<<<<<<<
+ * self.obj = obj
+ * self.flags = flags
+ */
+
+/* Python wrapper */
+static int __pyx_memoryview___cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
+static int __pyx_memoryview___cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
+ PyObject *__pyx_v_obj = 0;
+ int __pyx_v_flags;
+ int __pyx_v_dtype_is_object;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__cinit__ (wrapper)", 0);
+ {
+ static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_obj,&__pyx_n_s_flags,&__pyx_n_s_dtype_is_object,0};
+ PyObject* values[3] = {0,0,0};
+ if (unlikely(__pyx_kwds)) {
+ Py_ssize_t kw_args;
+ const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
+ switch (pos_args) {
+ case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
+ CYTHON_FALLTHROUGH;
+ case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ CYTHON_FALLTHROUGH;
+ case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ CYTHON_FALLTHROUGH;
+ case 0: break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ kw_args = PyDict_Size(__pyx_kwds);
+ switch (pos_args) {
+ case 0:
+ if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_obj)) != 0)) kw_args--;
+ else goto __pyx_L5_argtuple_error;
+ CYTHON_FALLTHROUGH;
+ case 1:
+ if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_flags)) != 0)) kw_args--;
+ else {
+ __Pyx_RaiseArgtupleInvalid("__cinit__", 0, 2, 3, 1); __PYX_ERR(1, 345, __pyx_L3_error)
+ }
+ CYTHON_FALLTHROUGH;
+ case 2:
+ if (kw_args > 0) {
+ PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_dtype_is_object);
+ if (value) { values[2] = value; kw_args--; }
+ }
+ }
+ if (unlikely(kw_args > 0)) {
+ if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "__cinit__") < 0)) __PYX_ERR(1, 345, __pyx_L3_error)
+ }
+ } else {
+ switch (PyTuple_GET_SIZE(__pyx_args)) {
+ case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
+ CYTHON_FALLTHROUGH;
+ case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
+ values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
+ break;
+ default: goto __pyx_L5_argtuple_error;
+ }
+ }
+ __pyx_v_obj = values[0];
+ __pyx_v_flags = __Pyx_PyInt_As_int(values[1]); if (unlikely((__pyx_v_flags == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 345, __pyx_L3_error)
+ if (values[2]) {
+ __pyx_v_dtype_is_object = __Pyx_PyObject_IsTrue(values[2]); if (unlikely((__pyx_v_dtype_is_object == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 345, __pyx_L3_error)
+ } else {
+ __pyx_v_dtype_is_object = ((int)0);
+ }
+ }
+ goto __pyx_L4_argument_unpacking_done;
+ __pyx_L5_argtuple_error:;
+ __Pyx_RaiseArgtupleInvalid("__cinit__", 0, 2, 3, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(1, 345, __pyx_L3_error)
+ __pyx_L3_error:;
+ __Pyx_AddTraceback("View.MemoryView.memoryview.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __Pyx_RefNannyFinishContext();
+ return -1;
+ __pyx_L4_argument_unpacking_done:;
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview___cinit__(((struct __pyx_memoryview_obj *)__pyx_v_self), __pyx_v_obj, __pyx_v_flags, __pyx_v_dtype_is_object);
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview___cinit__(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_obj, int __pyx_v_flags, int __pyx_v_dtype_is_object) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ int __pyx_t_3;
+ int __pyx_t_4;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__cinit__", 0);
+
+ /* "View.MemoryView":346
+ *
+ * def __cinit__(memoryview self, object obj, int flags, bint dtype_is_object=False):
+ * self.obj = obj # <<<<<<<<<<<<<<
+ * self.flags = flags
+ * if type(self) is memoryview or obj is not None:
+ */
+ __Pyx_INCREF(__pyx_v_obj);
+ __Pyx_GIVEREF(__pyx_v_obj);
+ __Pyx_GOTREF(__pyx_v_self->obj);
+ __Pyx_DECREF(__pyx_v_self->obj);
+ __pyx_v_self->obj = __pyx_v_obj;
+
+ /* "View.MemoryView":347
+ * def __cinit__(memoryview self, object obj, int flags, bint dtype_is_object=False):
+ * self.obj = obj
+ * self.flags = flags # <<<<<<<<<<<<<<
+ * if type(self) is memoryview or obj is not None:
+ * __Pyx_GetBuffer(obj, &self.view, flags)
+ */
+ __pyx_v_self->flags = __pyx_v_flags;
+
+ /* "View.MemoryView":348
+ * self.obj = obj
+ * self.flags = flags
+ * if type(self) is memoryview or obj is not None: # <<<<<<<<<<<<<<
+ * __Pyx_GetBuffer(obj, &self.view, flags)
+ * if self.view.obj == NULL:
+ */
+ __pyx_t_2 = (((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self))) == ((PyObject *)__pyx_memoryview_type));
+ __pyx_t_3 = (__pyx_t_2 != 0);
+ if (!__pyx_t_3) {
+ } else {
+ __pyx_t_1 = __pyx_t_3;
+ goto __pyx_L4_bool_binop_done;
+ }
+ __pyx_t_3 = (__pyx_v_obj != Py_None);
+ __pyx_t_2 = (__pyx_t_3 != 0);
+ __pyx_t_1 = __pyx_t_2;
+ __pyx_L4_bool_binop_done:;
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":349
+ * self.flags = flags
+ * if type(self) is memoryview or obj is not None:
+ * __Pyx_GetBuffer(obj, &self.view, flags) # <<<<<<<<<<<<<<
+ * if self.view.obj == NULL:
+ * (<__pyx_buffer *> &self.view).obj = Py_None
+ */
+ __pyx_t_4 = __Pyx_GetBuffer(__pyx_v_obj, (&__pyx_v_self->view), __pyx_v_flags); if (unlikely(__pyx_t_4 == ((int)-1))) __PYX_ERR(1, 349, __pyx_L1_error)
+
+ /* "View.MemoryView":350
+ * if type(self) is memoryview or obj is not None:
+ * __Pyx_GetBuffer(obj, &self.view, flags)
+ * if self.view.obj == NULL: # <<<<<<<<<<<<<<
+ * (<__pyx_buffer *> &self.view).obj = Py_None
+ * Py_INCREF(Py_None)
+ */
+ __pyx_t_1 = ((((PyObject *)__pyx_v_self->view.obj) == NULL) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":351
+ * __Pyx_GetBuffer(obj, &self.view, flags)
+ * if self.view.obj == NULL:
+ * (<__pyx_buffer *> &self.view).obj = Py_None # <<<<<<<<<<<<<<
+ * Py_INCREF(Py_None)
+ *
+ */
+ ((Py_buffer *)(&__pyx_v_self->view))->obj = Py_None;
+
+ /* "View.MemoryView":352
+ * if self.view.obj == NULL:
+ * (<__pyx_buffer *> &self.view).obj = Py_None
+ * Py_INCREF(Py_None) # <<<<<<<<<<<<<<
+ *
+ * global __pyx_memoryview_thread_locks_used
+ */
+ Py_INCREF(Py_None);
+
+ /* "View.MemoryView":350
+ * if type(self) is memoryview or obj is not None:
+ * __Pyx_GetBuffer(obj, &self.view, flags)
+ * if self.view.obj == NULL: # <<<<<<<<<<<<<<
+ * (<__pyx_buffer *> &self.view).obj = Py_None
+ * Py_INCREF(Py_None)
+ */
+ }
+
+ /* "View.MemoryView":348
+ * self.obj = obj
+ * self.flags = flags
+ * if type(self) is memoryview or obj is not None: # <<<<<<<<<<<<<<
+ * __Pyx_GetBuffer(obj, &self.view, flags)
+ * if self.view.obj == NULL:
+ */
+ }
+
+ /* "View.MemoryView":355
+ *
+ * global __pyx_memoryview_thread_locks_used
+ * if __pyx_memoryview_thread_locks_used < THREAD_LOCKS_PREALLOCATED: # <<<<<<<<<<<<<<
+ * self.lock = __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used]
+ * __pyx_memoryview_thread_locks_used += 1
+ */
+ __pyx_t_1 = ((__pyx_memoryview_thread_locks_used < 8) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":356
+ * global __pyx_memoryview_thread_locks_used
+ * if __pyx_memoryview_thread_locks_used < THREAD_LOCKS_PREALLOCATED:
+ * self.lock = __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used] # <<<<<<<<<<<<<<
+ * __pyx_memoryview_thread_locks_used += 1
+ * if self.lock is NULL:
+ */
+ __pyx_v_self->lock = (__pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used]);
+
+ /* "View.MemoryView":357
+ * if __pyx_memoryview_thread_locks_used < THREAD_LOCKS_PREALLOCATED:
+ * self.lock = __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used]
+ * __pyx_memoryview_thread_locks_used += 1 # <<<<<<<<<<<<<<
+ * if self.lock is NULL:
+ * self.lock = PyThread_allocate_lock()
+ */
+ __pyx_memoryview_thread_locks_used = (__pyx_memoryview_thread_locks_used + 1);
+
+ /* "View.MemoryView":355
+ *
+ * global __pyx_memoryview_thread_locks_used
+ * if __pyx_memoryview_thread_locks_used < THREAD_LOCKS_PREALLOCATED: # <<<<<<<<<<<<<<
+ * self.lock = __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used]
+ * __pyx_memoryview_thread_locks_used += 1
+ */
+ }
+
+ /* "View.MemoryView":358
+ * self.lock = __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used]
+ * __pyx_memoryview_thread_locks_used += 1
+ * if self.lock is NULL: # <<<<<<<<<<<<<<
+ * self.lock = PyThread_allocate_lock()
+ * if self.lock is NULL:
+ */
+ __pyx_t_1 = ((__pyx_v_self->lock == NULL) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":359
+ * __pyx_memoryview_thread_locks_used += 1
+ * if self.lock is NULL:
+ * self.lock = PyThread_allocate_lock() # <<<<<<<<<<<<<<
+ * if self.lock is NULL:
+ * raise MemoryError
+ */
+ __pyx_v_self->lock = PyThread_allocate_lock();
+
+ /* "View.MemoryView":360
+ * if self.lock is NULL:
+ * self.lock = PyThread_allocate_lock()
+ * if self.lock is NULL: # <<<<<<<<<<<<<<
+ * raise MemoryError
+ *
+ */
+ __pyx_t_1 = ((__pyx_v_self->lock == NULL) != 0);
+ if (unlikely(__pyx_t_1)) {
+
+ /* "View.MemoryView":361
+ * self.lock = PyThread_allocate_lock()
+ * if self.lock is NULL:
+ * raise MemoryError # <<<<<<<<<<<<<<
+ *
+ * if flags & PyBUF_FORMAT:
+ */
+ PyErr_NoMemory(); __PYX_ERR(1, 361, __pyx_L1_error)
+
+ /* "View.MemoryView":360
+ * if self.lock is NULL:
+ * self.lock = PyThread_allocate_lock()
+ * if self.lock is NULL: # <<<<<<<<<<<<<<
+ * raise MemoryError
+ *
+ */
+ }
+
+ /* "View.MemoryView":358
+ * self.lock = __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used]
+ * __pyx_memoryview_thread_locks_used += 1
+ * if self.lock is NULL: # <<<<<<<<<<<<<<
+ * self.lock = PyThread_allocate_lock()
+ * if self.lock is NULL:
+ */
+ }
+
+ /* "View.MemoryView":363
+ * raise MemoryError
+ *
+ * if flags & PyBUF_FORMAT: # <<<<<<<<<<<<<<
+ * self.dtype_is_object = (self.view.format[0] == b'O' and self.view.format[1] == b'\0')
+ * else:
+ */
+ __pyx_t_1 = ((__pyx_v_flags & PyBUF_FORMAT) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":364
+ *
+ * if flags & PyBUF_FORMAT:
+ * self.dtype_is_object = (self.view.format[0] == b'O' and self.view.format[1] == b'\0') # <<<<<<<<<<<<<<
+ * else:
+ * self.dtype_is_object = dtype_is_object
+ */
+ __pyx_t_2 = (((__pyx_v_self->view.format[0]) == 'O') != 0);
+ if (__pyx_t_2) {
+ } else {
+ __pyx_t_1 = __pyx_t_2;
+ goto __pyx_L11_bool_binop_done;
+ }
+ __pyx_t_2 = (((__pyx_v_self->view.format[1]) == '\x00') != 0);
+ __pyx_t_1 = __pyx_t_2;
+ __pyx_L11_bool_binop_done:;
+ __pyx_v_self->dtype_is_object = __pyx_t_1;
+
+ /* "View.MemoryView":363
+ * raise MemoryError
+ *
+ * if flags & PyBUF_FORMAT: # <<<<<<<<<<<<<<
+ * self.dtype_is_object = (self.view.format[0] == b'O' and self.view.format[1] == b'\0')
+ * else:
+ */
+ goto __pyx_L10;
+ }
+
+ /* "View.MemoryView":366
+ * self.dtype_is_object = (self.view.format[0] == b'O' and self.view.format[1] == b'\0')
+ * else:
+ * self.dtype_is_object = dtype_is_object # <<<<<<<<<<<<<<
+ *
+ * self.acquisition_count_aligned_p = <__pyx_atomic_int *> align_pointer(
+ */
+ /*else*/ {
+ __pyx_v_self->dtype_is_object = __pyx_v_dtype_is_object;
+ }
+ __pyx_L10:;
+
+ /* "View.MemoryView":368
+ * self.dtype_is_object = dtype_is_object
+ *
+ * self.acquisition_count_aligned_p = <__pyx_atomic_int *> align_pointer( # <<<<<<<<<<<<<<
+ * &self.acquisition_count[0], sizeof(__pyx_atomic_int))
+ * self.typeinfo = NULL
+ */
+ __pyx_v_self->acquisition_count_aligned_p = ((__pyx_atomic_int *)__pyx_align_pointer(((void *)(&(__pyx_v_self->acquisition_count[0]))), (sizeof(__pyx_atomic_int))));
+
+ /* "View.MemoryView":370
+ * self.acquisition_count_aligned_p = <__pyx_atomic_int *> align_pointer(
+ * &self.acquisition_count[0], sizeof(__pyx_atomic_int))
+ * self.typeinfo = NULL # <<<<<<<<<<<<<<
+ *
+ * def __dealloc__(memoryview self):
+ */
+ __pyx_v_self->typeinfo = NULL;
+
+ /* "View.MemoryView":345
+ * cdef __Pyx_TypeInfo *typeinfo
+ *
+ * def __cinit__(memoryview self, object obj, int flags, bint dtype_is_object=False): # <<<<<<<<<<<<<<
+ * self.obj = obj
+ * self.flags = flags
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_AddTraceback("View.MemoryView.memoryview.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = -1;
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":372
+ * self.typeinfo = NULL
+ *
+ * def __dealloc__(memoryview self): # <<<<<<<<<<<<<<
+ * if self.obj is not None:
+ * __Pyx_ReleaseBuffer(&self.view)
+ */
+
+/* Python wrapper */
+static void __pyx_memoryview___dealloc__(PyObject *__pyx_v_self); /*proto*/
+static void __pyx_memoryview___dealloc__(PyObject *__pyx_v_self) {
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__dealloc__ (wrapper)", 0);
+ __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_2__dealloc__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+}
+
+static void __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_2__dealloc__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ int __pyx_v_i;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ int __pyx_t_3;
+ int __pyx_t_4;
+ int __pyx_t_5;
+ PyThread_type_lock __pyx_t_6;
+ PyThread_type_lock __pyx_t_7;
+ __Pyx_RefNannySetupContext("__dealloc__", 0);
+
+ /* "View.MemoryView":373
+ *
+ * def __dealloc__(memoryview self):
+ * if self.obj is not None: # <<<<<<<<<<<<<<
+ * __Pyx_ReleaseBuffer(&self.view)
+ * elif (<__pyx_buffer *> &self.view).obj == Py_None:
+ */
+ __pyx_t_1 = (__pyx_v_self->obj != Py_None);
+ __pyx_t_2 = (__pyx_t_1 != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":374
+ * def __dealloc__(memoryview self):
+ * if self.obj is not None:
+ * __Pyx_ReleaseBuffer(&self.view) # <<<<<<<<<<<<<<
+ * elif (<__pyx_buffer *> &self.view).obj == Py_None:
+ *
+ */
+ __Pyx_ReleaseBuffer((&__pyx_v_self->view));
+
+ /* "View.MemoryView":373
+ *
+ * def __dealloc__(memoryview self):
+ * if self.obj is not None: # <<<<<<<<<<<<<<
+ * __Pyx_ReleaseBuffer(&self.view)
+ * elif (<__pyx_buffer *> &self.view).obj == Py_None:
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":375
+ * if self.obj is not None:
+ * __Pyx_ReleaseBuffer(&self.view)
+ * elif (<__pyx_buffer *> &self.view).obj == Py_None: # <<<<<<<<<<<<<<
+ *
+ * (<__pyx_buffer *> &self.view).obj = NULL
+ */
+ __pyx_t_2 = ((((Py_buffer *)(&__pyx_v_self->view))->obj == Py_None) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":377
+ * elif (<__pyx_buffer *> &self.view).obj == Py_None:
+ *
+ * (<__pyx_buffer *> &self.view).obj = NULL # <<<<<<<<<<<<<<
+ * Py_DECREF(Py_None)
+ *
+ */
+ ((Py_buffer *)(&__pyx_v_self->view))->obj = NULL;
+
+ /* "View.MemoryView":378
+ *
+ * (<__pyx_buffer *> &self.view).obj = NULL
+ * Py_DECREF(Py_None) # <<<<<<<<<<<<<<
+ *
+ * cdef int i
+ */
+ Py_DECREF(Py_None);
+
+ /* "View.MemoryView":375
+ * if self.obj is not None:
+ * __Pyx_ReleaseBuffer(&self.view)
+ * elif (<__pyx_buffer *> &self.view).obj == Py_None: # <<<<<<<<<<<<<<
+ *
+ * (<__pyx_buffer *> &self.view).obj = NULL
+ */
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":382
+ * cdef int i
+ * global __pyx_memoryview_thread_locks_used
+ * if self.lock != NULL: # <<<<<<<<<<<<<<
+ * for i in range(__pyx_memoryview_thread_locks_used):
+ * if __pyx_memoryview_thread_locks[i] is self.lock:
+ */
+ __pyx_t_2 = ((__pyx_v_self->lock != NULL) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":383
+ * global __pyx_memoryview_thread_locks_used
+ * if self.lock != NULL:
+ * for i in range(__pyx_memoryview_thread_locks_used): # <<<<<<<<<<<<<<
+ * if __pyx_memoryview_thread_locks[i] is self.lock:
+ * __pyx_memoryview_thread_locks_used -= 1
+ */
+ __pyx_t_3 = __pyx_memoryview_thread_locks_used;
+ __pyx_t_4 = __pyx_t_3;
+ for (__pyx_t_5 = 0; __pyx_t_5 < __pyx_t_4; __pyx_t_5+=1) {
+ __pyx_v_i = __pyx_t_5;
+
+ /* "View.MemoryView":384
+ * if self.lock != NULL:
+ * for i in range(__pyx_memoryview_thread_locks_used):
+ * if __pyx_memoryview_thread_locks[i] is self.lock: # <<<<<<<<<<<<<<
+ * __pyx_memoryview_thread_locks_used -= 1
+ * if i != __pyx_memoryview_thread_locks_used:
+ */
+ __pyx_t_2 = (((__pyx_memoryview_thread_locks[__pyx_v_i]) == __pyx_v_self->lock) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":385
+ * for i in range(__pyx_memoryview_thread_locks_used):
+ * if __pyx_memoryview_thread_locks[i] is self.lock:
+ * __pyx_memoryview_thread_locks_used -= 1 # <<<<<<<<<<<<<<
+ * if i != __pyx_memoryview_thread_locks_used:
+ * __pyx_memoryview_thread_locks[i], __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used] = (
+ */
+ __pyx_memoryview_thread_locks_used = (__pyx_memoryview_thread_locks_used - 1);
+
+ /* "View.MemoryView":386
+ * if __pyx_memoryview_thread_locks[i] is self.lock:
+ * __pyx_memoryview_thread_locks_used -= 1
+ * if i != __pyx_memoryview_thread_locks_used: # <<<<<<<<<<<<<<
+ * __pyx_memoryview_thread_locks[i], __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used] = (
+ * __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used], __pyx_memoryview_thread_locks[i])
+ */
+ __pyx_t_2 = ((__pyx_v_i != __pyx_memoryview_thread_locks_used) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":388
+ * if i != __pyx_memoryview_thread_locks_used:
+ * __pyx_memoryview_thread_locks[i], __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used] = (
+ * __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used], __pyx_memoryview_thread_locks[i]) # <<<<<<<<<<<<<<
+ * break
+ * else:
+ */
+ __pyx_t_6 = (__pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used]);
+ __pyx_t_7 = (__pyx_memoryview_thread_locks[__pyx_v_i]);
+
+ /* "View.MemoryView":387
+ * __pyx_memoryview_thread_locks_used -= 1
+ * if i != __pyx_memoryview_thread_locks_used:
+ * __pyx_memoryview_thread_locks[i], __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used] = ( # <<<<<<<<<<<<<<
+ * __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used], __pyx_memoryview_thread_locks[i])
+ * break
+ */
+ (__pyx_memoryview_thread_locks[__pyx_v_i]) = __pyx_t_6;
+ (__pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used]) = __pyx_t_7;
+
+ /* "View.MemoryView":386
+ * if __pyx_memoryview_thread_locks[i] is self.lock:
+ * __pyx_memoryview_thread_locks_used -= 1
+ * if i != __pyx_memoryview_thread_locks_used: # <<<<<<<<<<<<<<
+ * __pyx_memoryview_thread_locks[i], __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used] = (
+ * __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used], __pyx_memoryview_thread_locks[i])
+ */
+ }
+
+ /* "View.MemoryView":389
+ * __pyx_memoryview_thread_locks[i], __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used] = (
+ * __pyx_memoryview_thread_locks[__pyx_memoryview_thread_locks_used], __pyx_memoryview_thread_locks[i])
+ * break # <<<<<<<<<<<<<<
+ * else:
+ * PyThread_free_lock(self.lock)
+ */
+ goto __pyx_L6_break;
+
+ /* "View.MemoryView":384
+ * if self.lock != NULL:
+ * for i in range(__pyx_memoryview_thread_locks_used):
+ * if __pyx_memoryview_thread_locks[i] is self.lock: # <<<<<<<<<<<<<<
+ * __pyx_memoryview_thread_locks_used -= 1
+ * if i != __pyx_memoryview_thread_locks_used:
+ */
+ }
+ }
+ /*else*/ {
+
+ /* "View.MemoryView":391
+ * break
+ * else:
+ * PyThread_free_lock(self.lock) # <<<<<<<<<<<<<<
+ *
+ * cdef char *get_item_pointer(memoryview self, object index) except NULL:
+ */
+ PyThread_free_lock(__pyx_v_self->lock);
+ }
+ __pyx_L6_break:;
+
+ /* "View.MemoryView":382
+ * cdef int i
+ * global __pyx_memoryview_thread_locks_used
+ * if self.lock != NULL: # <<<<<<<<<<<<<<
+ * for i in range(__pyx_memoryview_thread_locks_used):
+ * if __pyx_memoryview_thread_locks[i] is self.lock:
+ */
+ }
+
+ /* "View.MemoryView":372
+ * self.typeinfo = NULL
+ *
+ * def __dealloc__(memoryview self): # <<<<<<<<<<<<<<
+ * if self.obj is not None:
+ * __Pyx_ReleaseBuffer(&self.view)
+ */
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+}
+
+/* "View.MemoryView":393
+ * PyThread_free_lock(self.lock)
+ *
+ * cdef char *get_item_pointer(memoryview self, object index) except NULL: # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t dim
+ * cdef char *itemp = self.view.buf
+ */
+
+static char *__pyx_memoryview_get_item_pointer(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_index) {
+ Py_ssize_t __pyx_v_dim;
+ char *__pyx_v_itemp;
+ PyObject *__pyx_v_idx = NULL;
+ char *__pyx_r;
+ __Pyx_RefNannyDeclarations
+ Py_ssize_t __pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ Py_ssize_t __pyx_t_3;
+ PyObject *(*__pyx_t_4)(PyObject *);
+ PyObject *__pyx_t_5 = NULL;
+ Py_ssize_t __pyx_t_6;
+ char *__pyx_t_7;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("get_item_pointer", 0);
+
+ /* "View.MemoryView":395
+ * cdef char *get_item_pointer(memoryview self, object index) except NULL:
+ * cdef Py_ssize_t dim
+ * cdef char *itemp = self.view.buf # <<<<<<<<<<<<<<
+ *
+ * for dim, idx in enumerate(index):
+ */
+ __pyx_v_itemp = ((char *)__pyx_v_self->view.buf);
+
+ /* "View.MemoryView":397
+ * cdef char *itemp = self.view.buf
+ *
+ * for dim, idx in enumerate(index): # <<<<<<<<<<<<<<
+ * itemp = pybuffer_index(&self.view, itemp, idx, dim)
+ *
+ */
+ __pyx_t_1 = 0;
+ if (likely(PyList_CheckExact(__pyx_v_index)) || PyTuple_CheckExact(__pyx_v_index)) {
+ __pyx_t_2 = __pyx_v_index; __Pyx_INCREF(__pyx_t_2); __pyx_t_3 = 0;
+ __pyx_t_4 = NULL;
+ } else {
+ __pyx_t_3 = -1; __pyx_t_2 = PyObject_GetIter(__pyx_v_index); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 397, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_4 = Py_TYPE(__pyx_t_2)->tp_iternext; if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 397, __pyx_L1_error)
+ }
+ for (;;) {
+ if (likely(!__pyx_t_4)) {
+ if (likely(PyList_CheckExact(__pyx_t_2))) {
+ if (__pyx_t_3 >= PyList_GET_SIZE(__pyx_t_2)) break;
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_5 = PyList_GET_ITEM(__pyx_t_2, __pyx_t_3); __Pyx_INCREF(__pyx_t_5); __pyx_t_3++; if (unlikely(0 < 0)) __PYX_ERR(1, 397, __pyx_L1_error)
+ #else
+ __pyx_t_5 = PySequence_ITEM(__pyx_t_2, __pyx_t_3); __pyx_t_3++; if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 397, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ #endif
+ } else {
+ if (__pyx_t_3 >= PyTuple_GET_SIZE(__pyx_t_2)) break;
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_5 = PyTuple_GET_ITEM(__pyx_t_2, __pyx_t_3); __Pyx_INCREF(__pyx_t_5); __pyx_t_3++; if (unlikely(0 < 0)) __PYX_ERR(1, 397, __pyx_L1_error)
+ #else
+ __pyx_t_5 = PySequence_ITEM(__pyx_t_2, __pyx_t_3); __pyx_t_3++; if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 397, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ #endif
+ }
+ } else {
+ __pyx_t_5 = __pyx_t_4(__pyx_t_2);
+ if (unlikely(!__pyx_t_5)) {
+ PyObject* exc_type = PyErr_Occurred();
+ if (exc_type) {
+ if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear();
+ else __PYX_ERR(1, 397, __pyx_L1_error)
+ }
+ break;
+ }
+ __Pyx_GOTREF(__pyx_t_5);
+ }
+ __Pyx_XDECREF_SET(__pyx_v_idx, __pyx_t_5);
+ __pyx_t_5 = 0;
+ __pyx_v_dim = __pyx_t_1;
+ __pyx_t_1 = (__pyx_t_1 + 1);
+
+ /* "View.MemoryView":398
+ *
+ * for dim, idx in enumerate(index):
+ * itemp = pybuffer_index(&self.view, itemp, idx, dim) # <<<<<<<<<<<<<<
+ *
+ * return itemp
+ */
+ __pyx_t_6 = __Pyx_PyIndex_AsSsize_t(__pyx_v_idx); if (unlikely((__pyx_t_6 == (Py_ssize_t)-1) && PyErr_Occurred())) __PYX_ERR(1, 398, __pyx_L1_error)
+ __pyx_t_7 = __pyx_pybuffer_index((&__pyx_v_self->view), __pyx_v_itemp, __pyx_t_6, __pyx_v_dim); if (unlikely(__pyx_t_7 == ((char *)NULL))) __PYX_ERR(1, 398, __pyx_L1_error)
+ __pyx_v_itemp = __pyx_t_7;
+
+ /* "View.MemoryView":397
+ * cdef char *itemp = self.view.buf
+ *
+ * for dim, idx in enumerate(index): # <<<<<<<<<<<<<<
+ * itemp = pybuffer_index(&self.view, itemp, idx, dim)
+ *
+ */
+ }
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+
+ /* "View.MemoryView":400
+ * itemp = pybuffer_index(&self.view, itemp, idx, dim)
+ *
+ * return itemp # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_r = __pyx_v_itemp;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":393
+ * PyThread_free_lock(self.lock)
+ *
+ * cdef char *get_item_pointer(memoryview self, object index) except NULL: # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t dim
+ * cdef char *itemp = self.view.buf
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.get_item_pointer", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_idx);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":403
+ *
+ *
+ * def __getitem__(memoryview self, object index): # <<<<<<<<<<<<<<
+ * if index is Ellipsis:
+ * return self
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_memoryview___getitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_index); /*proto*/
+static PyObject *__pyx_memoryview___getitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_index) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__getitem__ (wrapper)", 0);
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_4__getitem__(((struct __pyx_memoryview_obj *)__pyx_v_self), ((PyObject *)__pyx_v_index));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_4__getitem__(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_index) {
+ PyObject *__pyx_v_have_slices = NULL;
+ PyObject *__pyx_v_indices = NULL;
+ char *__pyx_v_itemp;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ PyObject *__pyx_t_4 = NULL;
+ PyObject *__pyx_t_5 = NULL;
+ char *__pyx_t_6;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__getitem__", 0);
+
+ /* "View.MemoryView":404
+ *
+ * def __getitem__(memoryview self, object index):
+ * if index is Ellipsis: # <<<<<<<<<<<<<<
+ * return self
+ *
+ */
+ __pyx_t_1 = (__pyx_v_index == __pyx_builtin_Ellipsis);
+ __pyx_t_2 = (__pyx_t_1 != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":405
+ * def __getitem__(memoryview self, object index):
+ * if index is Ellipsis:
+ * return self # <<<<<<<<<<<<<<
+ *
+ * have_slices, indices = _unellipsify(index, self.view.ndim)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(((PyObject *)__pyx_v_self));
+ __pyx_r = ((PyObject *)__pyx_v_self);
+ goto __pyx_L0;
+
+ /* "View.MemoryView":404
+ *
+ * def __getitem__(memoryview self, object index):
+ * if index is Ellipsis: # <<<<<<<<<<<<<<
+ * return self
+ *
+ */
+ }
+
+ /* "View.MemoryView":407
+ * return self
+ *
+ * have_slices, indices = _unellipsify(index, self.view.ndim) # <<<<<<<<<<<<<<
+ *
+ * cdef char *itemp
+ */
+ __pyx_t_3 = _unellipsify(__pyx_v_index, __pyx_v_self->view.ndim); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 407, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ if (likely(__pyx_t_3 != Py_None)) {
+ PyObject* sequence = __pyx_t_3;
+ Py_ssize_t size = __Pyx_PySequence_SIZE(sequence);
+ if (unlikely(size != 2)) {
+ if (size > 2) __Pyx_RaiseTooManyValuesError(2);
+ else if (size >= 0) __Pyx_RaiseNeedMoreValuesError(size);
+ __PYX_ERR(1, 407, __pyx_L1_error)
+ }
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_4 = PyTuple_GET_ITEM(sequence, 0);
+ __pyx_t_5 = PyTuple_GET_ITEM(sequence, 1);
+ __Pyx_INCREF(__pyx_t_4);
+ __Pyx_INCREF(__pyx_t_5);
+ #else
+ __pyx_t_4 = PySequence_ITEM(sequence, 0); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 407, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_5 = PySequence_ITEM(sequence, 1); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 407, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ #endif
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ } else {
+ __Pyx_RaiseNoneNotIterableError(); __PYX_ERR(1, 407, __pyx_L1_error)
+ }
+ __pyx_v_have_slices = __pyx_t_4;
+ __pyx_t_4 = 0;
+ __pyx_v_indices = __pyx_t_5;
+ __pyx_t_5 = 0;
+
+ /* "View.MemoryView":410
+ *
+ * cdef char *itemp
+ * if have_slices: # <<<<<<<<<<<<<<
+ * return memview_slice(self, indices)
+ * else:
+ */
+ __pyx_t_2 = __Pyx_PyObject_IsTrue(__pyx_v_have_slices); if (unlikely(__pyx_t_2 < 0)) __PYX_ERR(1, 410, __pyx_L1_error)
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":411
+ * cdef char *itemp
+ * if have_slices:
+ * return memview_slice(self, indices) # <<<<<<<<<<<<<<
+ * else:
+ * itemp = self.get_item_pointer(indices)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_3 = ((PyObject *)__pyx_memview_slice(__pyx_v_self, __pyx_v_indices)); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 411, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_r = __pyx_t_3;
+ __pyx_t_3 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":410
+ *
+ * cdef char *itemp
+ * if have_slices: # <<<<<<<<<<<<<<
+ * return memview_slice(self, indices)
+ * else:
+ */
+ }
+
+ /* "View.MemoryView":413
+ * return memview_slice(self, indices)
+ * else:
+ * itemp = self.get_item_pointer(indices) # <<<<<<<<<<<<<<
+ * return self.convert_item_to_object(itemp)
+ *
+ */
+ /*else*/ {
+ __pyx_t_6 = ((struct __pyx_vtabstruct_memoryview *)__pyx_v_self->__pyx_vtab)->get_item_pointer(__pyx_v_self, __pyx_v_indices); if (unlikely(__pyx_t_6 == ((char *)NULL))) __PYX_ERR(1, 413, __pyx_L1_error)
+ __pyx_v_itemp = __pyx_t_6;
+
+ /* "View.MemoryView":414
+ * else:
+ * itemp = self.get_item_pointer(indices)
+ * return self.convert_item_to_object(itemp) # <<<<<<<<<<<<<<
+ *
+ * def __setitem__(memoryview self, object index, object value):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_3 = ((struct __pyx_vtabstruct_memoryview *)__pyx_v_self->__pyx_vtab)->convert_item_to_object(__pyx_v_self, __pyx_v_itemp); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 414, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_r = __pyx_t_3;
+ __pyx_t_3 = 0;
+ goto __pyx_L0;
+ }
+
+ /* "View.MemoryView":403
+ *
+ *
+ * def __getitem__(memoryview self, object index): # <<<<<<<<<<<<<<
+ * if index is Ellipsis:
+ * return self
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.__getitem__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_have_slices);
+ __Pyx_XDECREF(__pyx_v_indices);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":416
+ * return self.convert_item_to_object(itemp)
+ *
+ * def __setitem__(memoryview self, object index, object value): # <<<<<<<<<<<<<<
+ * if self.view.readonly:
+ * raise TypeError("Cannot assign to read-only memoryview")
+ */
+
+/* Python wrapper */
+static int __pyx_memoryview___setitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_index, PyObject *__pyx_v_value); /*proto*/
+static int __pyx_memoryview___setitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_index, PyObject *__pyx_v_value) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__setitem__ (wrapper)", 0);
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_6__setitem__(((struct __pyx_memoryview_obj *)__pyx_v_self), ((PyObject *)__pyx_v_index), ((PyObject *)__pyx_v_value));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_6__setitem__(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_index, PyObject *__pyx_v_value) {
+ PyObject *__pyx_v_have_slices = NULL;
+ PyObject *__pyx_v_obj = NULL;
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ PyObject *__pyx_t_4 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__setitem__", 0);
+ __Pyx_INCREF(__pyx_v_index);
+
+ /* "View.MemoryView":417
+ *
+ * def __setitem__(memoryview self, object index, object value):
+ * if self.view.readonly: # <<<<<<<<<<<<<<
+ * raise TypeError("Cannot assign to read-only memoryview")
+ *
+ */
+ __pyx_t_1 = (__pyx_v_self->view.readonly != 0);
+ if (unlikely(__pyx_t_1)) {
+
+ /* "View.MemoryView":418
+ * def __setitem__(memoryview self, object index, object value):
+ * if self.view.readonly:
+ * raise TypeError("Cannot assign to read-only memoryview") # <<<<<<<<<<<<<<
+ *
+ * have_slices, index = _unellipsify(index, self.view.ndim)
+ */
+ __pyx_t_2 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__12, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 418, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_Raise(__pyx_t_2, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __PYX_ERR(1, 418, __pyx_L1_error)
+
+ /* "View.MemoryView":417
+ *
+ * def __setitem__(memoryview self, object index, object value):
+ * if self.view.readonly: # <<<<<<<<<<<<<<
+ * raise TypeError("Cannot assign to read-only memoryview")
+ *
+ */
+ }
+
+ /* "View.MemoryView":420
+ * raise TypeError("Cannot assign to read-only memoryview")
+ *
+ * have_slices, index = _unellipsify(index, self.view.ndim) # <<<<<<<<<<<<<<
+ *
+ * if have_slices:
+ */
+ __pyx_t_2 = _unellipsify(__pyx_v_index, __pyx_v_self->view.ndim); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 420, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ if (likely(__pyx_t_2 != Py_None)) {
+ PyObject* sequence = __pyx_t_2;
+ Py_ssize_t size = __Pyx_PySequence_SIZE(sequence);
+ if (unlikely(size != 2)) {
+ if (size > 2) __Pyx_RaiseTooManyValuesError(2);
+ else if (size >= 0) __Pyx_RaiseNeedMoreValuesError(size);
+ __PYX_ERR(1, 420, __pyx_L1_error)
+ }
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_3 = PyTuple_GET_ITEM(sequence, 0);
+ __pyx_t_4 = PyTuple_GET_ITEM(sequence, 1);
+ __Pyx_INCREF(__pyx_t_3);
+ __Pyx_INCREF(__pyx_t_4);
+ #else
+ __pyx_t_3 = PySequence_ITEM(sequence, 0); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 420, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_t_4 = PySequence_ITEM(sequence, 1); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 420, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ #endif
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ } else {
+ __Pyx_RaiseNoneNotIterableError(); __PYX_ERR(1, 420, __pyx_L1_error)
+ }
+ __pyx_v_have_slices = __pyx_t_3;
+ __pyx_t_3 = 0;
+ __Pyx_DECREF_SET(__pyx_v_index, __pyx_t_4);
+ __pyx_t_4 = 0;
+
+ /* "View.MemoryView":422
+ * have_slices, index = _unellipsify(index, self.view.ndim)
+ *
+ * if have_slices: # <<<<<<<<<<<<<<
+ * obj = self.is_slice(value)
+ * if obj:
+ */
+ __pyx_t_1 = __Pyx_PyObject_IsTrue(__pyx_v_have_slices); if (unlikely(__pyx_t_1 < 0)) __PYX_ERR(1, 422, __pyx_L1_error)
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":423
+ *
+ * if have_slices:
+ * obj = self.is_slice(value) # <<<<<<<<<<<<<<
+ * if obj:
+ * self.setitem_slice_assignment(self[index], obj)
+ */
+ __pyx_t_2 = ((struct __pyx_vtabstruct_memoryview *)__pyx_v_self->__pyx_vtab)->is_slice(__pyx_v_self, __pyx_v_value); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 423, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_v_obj = __pyx_t_2;
+ __pyx_t_2 = 0;
+
+ /* "View.MemoryView":424
+ * if have_slices:
+ * obj = self.is_slice(value)
+ * if obj: # <<<<<<<<<<<<<<
+ * self.setitem_slice_assignment(self[index], obj)
+ * else:
+ */
+ __pyx_t_1 = __Pyx_PyObject_IsTrue(__pyx_v_obj); if (unlikely(__pyx_t_1 < 0)) __PYX_ERR(1, 424, __pyx_L1_error)
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":425
+ * obj = self.is_slice(value)
+ * if obj:
+ * self.setitem_slice_assignment(self[index], obj) # <<<<<<<<<<<<<<
+ * else:
+ * self.setitem_slice_assign_scalar(self[index], value)
+ */
+ __pyx_t_2 = __Pyx_PyObject_GetItem(((PyObject *)__pyx_v_self), __pyx_v_index); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 425, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_4 = ((struct __pyx_vtabstruct_memoryview *)__pyx_v_self->__pyx_vtab)->setitem_slice_assignment(__pyx_v_self, __pyx_t_2, __pyx_v_obj); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 425, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+
+ /* "View.MemoryView":424
+ * if have_slices:
+ * obj = self.is_slice(value)
+ * if obj: # <<<<<<<<<<<<<<
+ * self.setitem_slice_assignment(self[index], obj)
+ * else:
+ */
+ goto __pyx_L5;
+ }
+
+ /* "View.MemoryView":427
+ * self.setitem_slice_assignment(self[index], obj)
+ * else:
+ * self.setitem_slice_assign_scalar(self[index], value) # <<<<<<<<<<<<<<
+ * else:
+ * self.setitem_indexed(index, value)
+ */
+ /*else*/ {
+ __pyx_t_4 = __Pyx_PyObject_GetItem(((PyObject *)__pyx_v_self), __pyx_v_index); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 427, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ if (!(likely(((__pyx_t_4) == Py_None) || likely(__Pyx_TypeTest(__pyx_t_4, __pyx_memoryview_type))))) __PYX_ERR(1, 427, __pyx_L1_error)
+ __pyx_t_2 = ((struct __pyx_vtabstruct_memoryview *)__pyx_v_self->__pyx_vtab)->setitem_slice_assign_scalar(__pyx_v_self, ((struct __pyx_memoryview_obj *)__pyx_t_4), __pyx_v_value); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 427, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ }
+ __pyx_L5:;
+
+ /* "View.MemoryView":422
+ * have_slices, index = _unellipsify(index, self.view.ndim)
+ *
+ * if have_slices: # <<<<<<<<<<<<<<
+ * obj = self.is_slice(value)
+ * if obj:
+ */
+ goto __pyx_L4;
+ }
+
+ /* "View.MemoryView":429
+ * self.setitem_slice_assign_scalar(self[index], value)
+ * else:
+ * self.setitem_indexed(index, value) # <<<<<<<<<<<<<<
+ *
+ * cdef is_slice(self, obj):
+ */
+ /*else*/ {
+ __pyx_t_2 = ((struct __pyx_vtabstruct_memoryview *)__pyx_v_self->__pyx_vtab)->setitem_indexed(__pyx_v_self, __pyx_v_index, __pyx_v_value); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 429, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ }
+ __pyx_L4:;
+
+ /* "View.MemoryView":416
+ * return self.convert_item_to_object(itemp)
+ *
+ * def __setitem__(memoryview self, object index, object value): # <<<<<<<<<<<<<<
+ * if self.view.readonly:
+ * raise TypeError("Cannot assign to read-only memoryview")
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.__setitem__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = -1;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_have_slices);
+ __Pyx_XDECREF(__pyx_v_obj);
+ __Pyx_XDECREF(__pyx_v_index);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":431
+ * self.setitem_indexed(index, value)
+ *
+ * cdef is_slice(self, obj): # <<<<<<<<<<<<<<
+ * if not isinstance(obj, memoryview):
+ * try:
+ */
+
+static PyObject *__pyx_memoryview_is_slice(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_obj) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ PyObject *__pyx_t_4 = NULL;
+ PyObject *__pyx_t_5 = NULL;
+ PyObject *__pyx_t_6 = NULL;
+ PyObject *__pyx_t_7 = NULL;
+ PyObject *__pyx_t_8 = NULL;
+ int __pyx_t_9;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("is_slice", 0);
+ __Pyx_INCREF(__pyx_v_obj);
+
+ /* "View.MemoryView":432
+ *
+ * cdef is_slice(self, obj):
+ * if not isinstance(obj, memoryview): # <<<<<<<<<<<<<<
+ * try:
+ * obj = memoryview(obj, self.flags & ~PyBUF_WRITABLE | PyBUF_ANY_CONTIGUOUS,
+ */
+ __pyx_t_1 = __Pyx_TypeCheck(__pyx_v_obj, __pyx_memoryview_type);
+ __pyx_t_2 = ((!(__pyx_t_1 != 0)) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":433
+ * cdef is_slice(self, obj):
+ * if not isinstance(obj, memoryview):
+ * try: # <<<<<<<<<<<<<<
+ * obj = memoryview(obj, self.flags & ~PyBUF_WRITABLE | PyBUF_ANY_CONTIGUOUS,
+ * self.dtype_is_object)
+ */
+ {
+ __Pyx_PyThreadState_declare
+ __Pyx_PyThreadState_assign
+ __Pyx_ExceptionSave(&__pyx_t_3, &__pyx_t_4, &__pyx_t_5);
+ __Pyx_XGOTREF(__pyx_t_3);
+ __Pyx_XGOTREF(__pyx_t_4);
+ __Pyx_XGOTREF(__pyx_t_5);
+ /*try:*/ {
+
+ /* "View.MemoryView":434
+ * if not isinstance(obj, memoryview):
+ * try:
+ * obj = memoryview(obj, self.flags & ~PyBUF_WRITABLE | PyBUF_ANY_CONTIGUOUS, # <<<<<<<<<<<<<<
+ * self.dtype_is_object)
+ * except TypeError:
+ */
+ __pyx_t_6 = __Pyx_PyInt_From_int(((__pyx_v_self->flags & (~PyBUF_WRITABLE)) | PyBUF_ANY_CONTIGUOUS)); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 434, __pyx_L4_error)
+ __Pyx_GOTREF(__pyx_t_6);
+
+ /* "View.MemoryView":435
+ * try:
+ * obj = memoryview(obj, self.flags & ~PyBUF_WRITABLE | PyBUF_ANY_CONTIGUOUS,
+ * self.dtype_is_object) # <<<<<<<<<<<<<<
+ * except TypeError:
+ * return None
+ */
+ __pyx_t_7 = __Pyx_PyBool_FromLong(__pyx_v_self->dtype_is_object); if (unlikely(!__pyx_t_7)) __PYX_ERR(1, 435, __pyx_L4_error)
+ __Pyx_GOTREF(__pyx_t_7);
+
+ /* "View.MemoryView":434
+ * if not isinstance(obj, memoryview):
+ * try:
+ * obj = memoryview(obj, self.flags & ~PyBUF_WRITABLE | PyBUF_ANY_CONTIGUOUS, # <<<<<<<<<<<<<<
+ * self.dtype_is_object)
+ * except TypeError:
+ */
+ __pyx_t_8 = PyTuple_New(3); if (unlikely(!__pyx_t_8)) __PYX_ERR(1, 434, __pyx_L4_error)
+ __Pyx_GOTREF(__pyx_t_8);
+ __Pyx_INCREF(__pyx_v_obj);
+ __Pyx_GIVEREF(__pyx_v_obj);
+ PyTuple_SET_ITEM(__pyx_t_8, 0, __pyx_v_obj);
+ __Pyx_GIVEREF(__pyx_t_6);
+ PyTuple_SET_ITEM(__pyx_t_8, 1, __pyx_t_6);
+ __Pyx_GIVEREF(__pyx_t_7);
+ PyTuple_SET_ITEM(__pyx_t_8, 2, __pyx_t_7);
+ __pyx_t_6 = 0;
+ __pyx_t_7 = 0;
+ __pyx_t_7 = __Pyx_PyObject_Call(((PyObject *)__pyx_memoryview_type), __pyx_t_8, NULL); if (unlikely(!__pyx_t_7)) __PYX_ERR(1, 434, __pyx_L4_error)
+ __Pyx_GOTREF(__pyx_t_7);
+ __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
+ __Pyx_DECREF_SET(__pyx_v_obj, __pyx_t_7);
+ __pyx_t_7 = 0;
+
+ /* "View.MemoryView":433
+ * cdef is_slice(self, obj):
+ * if not isinstance(obj, memoryview):
+ * try: # <<<<<<<<<<<<<<
+ * obj = memoryview(obj, self.flags & ~PyBUF_WRITABLE | PyBUF_ANY_CONTIGUOUS,
+ * self.dtype_is_object)
+ */
+ }
+ __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0;
+ __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0;
+ goto __pyx_L9_try_end;
+ __pyx_L4_error:;
+ __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0;
+ __Pyx_XDECREF(__pyx_t_7); __pyx_t_7 = 0;
+ __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0;
+
+ /* "View.MemoryView":436
+ * obj = memoryview(obj, self.flags & ~PyBUF_WRITABLE | PyBUF_ANY_CONTIGUOUS,
+ * self.dtype_is_object)
+ * except TypeError: # <<<<<<<<<<<<<<
+ * return None
+ *
+ */
+ __pyx_t_9 = __Pyx_PyErr_ExceptionMatches(__pyx_builtin_TypeError);
+ if (__pyx_t_9) {
+ __Pyx_AddTraceback("View.MemoryView.memoryview.is_slice", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ if (__Pyx_GetException(&__pyx_t_7, &__pyx_t_8, &__pyx_t_6) < 0) __PYX_ERR(1, 436, __pyx_L6_except_error)
+ __Pyx_GOTREF(__pyx_t_7);
+ __Pyx_GOTREF(__pyx_t_8);
+ __Pyx_GOTREF(__pyx_t_6);
+
+ /* "View.MemoryView":437
+ * self.dtype_is_object)
+ * except TypeError:
+ * return None # <<<<<<<<<<<<<<
+ *
+ * return obj
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
+ __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0;
+ __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
+ goto __pyx_L7_except_return;
+ }
+ goto __pyx_L6_except_error;
+ __pyx_L6_except_error:;
+
+ /* "View.MemoryView":433
+ * cdef is_slice(self, obj):
+ * if not isinstance(obj, memoryview):
+ * try: # <<<<<<<<<<<<<<
+ * obj = memoryview(obj, self.flags & ~PyBUF_WRITABLE | PyBUF_ANY_CONTIGUOUS,
+ * self.dtype_is_object)
+ */
+ __Pyx_XGIVEREF(__pyx_t_3);
+ __Pyx_XGIVEREF(__pyx_t_4);
+ __Pyx_XGIVEREF(__pyx_t_5);
+ __Pyx_ExceptionReset(__pyx_t_3, __pyx_t_4, __pyx_t_5);
+ goto __pyx_L1_error;
+ __pyx_L7_except_return:;
+ __Pyx_XGIVEREF(__pyx_t_3);
+ __Pyx_XGIVEREF(__pyx_t_4);
+ __Pyx_XGIVEREF(__pyx_t_5);
+ __Pyx_ExceptionReset(__pyx_t_3, __pyx_t_4, __pyx_t_5);
+ goto __pyx_L0;
+ __pyx_L9_try_end:;
+ }
+
+ /* "View.MemoryView":432
+ *
+ * cdef is_slice(self, obj):
+ * if not isinstance(obj, memoryview): # <<<<<<<<<<<<<<
+ * try:
+ * obj = memoryview(obj, self.flags & ~PyBUF_WRITABLE | PyBUF_ANY_CONTIGUOUS,
+ */
+ }
+
+ /* "View.MemoryView":439
+ * return None
+ *
+ * return obj # <<<<<<<<<<<<<<
+ *
+ * cdef setitem_slice_assignment(self, dst, src):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(__pyx_v_obj);
+ __pyx_r = __pyx_v_obj;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":431
+ * self.setitem_indexed(index, value)
+ *
+ * cdef is_slice(self, obj): # <<<<<<<<<<<<<<
+ * if not isinstance(obj, memoryview):
+ * try:
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_6);
+ __Pyx_XDECREF(__pyx_t_7);
+ __Pyx_XDECREF(__pyx_t_8);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.is_slice", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_obj);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":441
+ * return obj
+ *
+ * cdef setitem_slice_assignment(self, dst, src): # <<<<<<<<<<<<<<
+ * cdef __Pyx_memviewslice dst_slice
+ * cdef __Pyx_memviewslice src_slice
+ */
+
+static PyObject *__pyx_memoryview_setitem_slice_assignment(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_dst, PyObject *__pyx_v_src) {
+ __Pyx_memviewslice __pyx_v_dst_slice;
+ __Pyx_memviewslice __pyx_v_src_slice;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ __Pyx_memviewslice *__pyx_t_1;
+ __Pyx_memviewslice *__pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_t_4;
+ int __pyx_t_5;
+ int __pyx_t_6;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("setitem_slice_assignment", 0);
+
+ /* "View.MemoryView":445
+ * cdef __Pyx_memviewslice src_slice
+ *
+ * memoryview_copy_contents(get_slice_from_memview(src, &src_slice)[0], # <<<<<<<<<<<<<<
+ * get_slice_from_memview(dst, &dst_slice)[0],
+ * src.ndim, dst.ndim, self.dtype_is_object)
+ */
+ if (!(likely(((__pyx_v_src) == Py_None) || likely(__Pyx_TypeTest(__pyx_v_src, __pyx_memoryview_type))))) __PYX_ERR(1, 445, __pyx_L1_error)
+ __pyx_t_1 = __pyx_memoryview_get_slice_from_memoryview(((struct __pyx_memoryview_obj *)__pyx_v_src), (&__pyx_v_src_slice)); if (unlikely(__pyx_t_1 == ((__Pyx_memviewslice *)NULL))) __PYX_ERR(1, 445, __pyx_L1_error)
+
+ /* "View.MemoryView":446
+ *
+ * memoryview_copy_contents(get_slice_from_memview(src, &src_slice)[0],
+ * get_slice_from_memview(dst, &dst_slice)[0], # <<<<<<<<<<<<<<
+ * src.ndim, dst.ndim, self.dtype_is_object)
+ *
+ */
+ if (!(likely(((__pyx_v_dst) == Py_None) || likely(__Pyx_TypeTest(__pyx_v_dst, __pyx_memoryview_type))))) __PYX_ERR(1, 446, __pyx_L1_error)
+ __pyx_t_2 = __pyx_memoryview_get_slice_from_memoryview(((struct __pyx_memoryview_obj *)__pyx_v_dst), (&__pyx_v_dst_slice)); if (unlikely(__pyx_t_2 == ((__Pyx_memviewslice *)NULL))) __PYX_ERR(1, 446, __pyx_L1_error)
+
+ /* "View.MemoryView":447
+ * memoryview_copy_contents(get_slice_from_memview(src, &src_slice)[0],
+ * get_slice_from_memview(dst, &dst_slice)[0],
+ * src.ndim, dst.ndim, self.dtype_is_object) # <<<<<<<<<<<<<<
+ *
+ * cdef setitem_slice_assign_scalar(self, memoryview dst, value):
+ */
+ __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_v_src, __pyx_n_s_ndim); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 447, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_t_4 = __Pyx_PyInt_As_int(__pyx_t_3); if (unlikely((__pyx_t_4 == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 447, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_v_dst, __pyx_n_s_ndim); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 447, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_t_5 = __Pyx_PyInt_As_int(__pyx_t_3); if (unlikely((__pyx_t_5 == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 447, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+
+ /* "View.MemoryView":445
+ * cdef __Pyx_memviewslice src_slice
+ *
+ * memoryview_copy_contents(get_slice_from_memview(src, &src_slice)[0], # <<<<<<<<<<<<<<
+ * get_slice_from_memview(dst, &dst_slice)[0],
+ * src.ndim, dst.ndim, self.dtype_is_object)
+ */
+ __pyx_t_6 = __pyx_memoryview_copy_contents((__pyx_t_1[0]), (__pyx_t_2[0]), __pyx_t_4, __pyx_t_5, __pyx_v_self->dtype_is_object); if (unlikely(__pyx_t_6 == ((int)-1))) __PYX_ERR(1, 445, __pyx_L1_error)
+
+ /* "View.MemoryView":441
+ * return obj
+ *
+ * cdef setitem_slice_assignment(self, dst, src): # <<<<<<<<<<<<<<
+ * cdef __Pyx_memviewslice dst_slice
+ * cdef __Pyx_memviewslice src_slice
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.setitem_slice_assignment", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":449
+ * src.ndim, dst.ndim, self.dtype_is_object)
+ *
+ * cdef setitem_slice_assign_scalar(self, memoryview dst, value): # <<<<<<<<<<<<<<
+ * cdef int array[128]
+ * cdef void *tmp = NULL
+ */
+
+static PyObject *__pyx_memoryview_setitem_slice_assign_scalar(struct __pyx_memoryview_obj *__pyx_v_self, struct __pyx_memoryview_obj *__pyx_v_dst, PyObject *__pyx_v_value) {
+ int __pyx_v_array[0x80];
+ void *__pyx_v_tmp;
+ void *__pyx_v_item;
+ __Pyx_memviewslice *__pyx_v_dst_slice;
+ __Pyx_memviewslice __pyx_v_tmp_slice;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ __Pyx_memviewslice *__pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_t_4;
+ int __pyx_t_5;
+ char const *__pyx_t_6;
+ PyObject *__pyx_t_7 = NULL;
+ PyObject *__pyx_t_8 = NULL;
+ PyObject *__pyx_t_9 = NULL;
+ PyObject *__pyx_t_10 = NULL;
+ PyObject *__pyx_t_11 = NULL;
+ PyObject *__pyx_t_12 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("setitem_slice_assign_scalar", 0);
+
+ /* "View.MemoryView":451
+ * cdef setitem_slice_assign_scalar(self, memoryview dst, value):
+ * cdef int array[128]
+ * cdef void *tmp = NULL # <<<<<<<<<<<<<<
+ * cdef void *item
+ *
+ */
+ __pyx_v_tmp = NULL;
+
+ /* "View.MemoryView":456
+ * cdef __Pyx_memviewslice *dst_slice
+ * cdef __Pyx_memviewslice tmp_slice
+ * dst_slice = get_slice_from_memview(dst, &tmp_slice) # <<<<<<<<<<<<<<
+ *
+ * if self.view.itemsize > sizeof(array):
+ */
+ __pyx_t_1 = __pyx_memoryview_get_slice_from_memoryview(__pyx_v_dst, (&__pyx_v_tmp_slice)); if (unlikely(__pyx_t_1 == ((__Pyx_memviewslice *)NULL))) __PYX_ERR(1, 456, __pyx_L1_error)
+ __pyx_v_dst_slice = __pyx_t_1;
+
+ /* "View.MemoryView":458
+ * dst_slice = get_slice_from_memview(dst, &tmp_slice)
+ *
+ * if self.view.itemsize > sizeof(array): # <<<<<<<<<<<<<<
+ * tmp = PyMem_Malloc(self.view.itemsize)
+ * if tmp == NULL:
+ */
+ __pyx_t_2 = ((((size_t)__pyx_v_self->view.itemsize) > (sizeof(__pyx_v_array))) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":459
+ *
+ * if self.view.itemsize > sizeof(array):
+ * tmp = PyMem_Malloc(self.view.itemsize) # <<<<<<<<<<<<<<
+ * if tmp == NULL:
+ * raise MemoryError
+ */
+ __pyx_v_tmp = PyMem_Malloc(__pyx_v_self->view.itemsize);
+
+ /* "View.MemoryView":460
+ * if self.view.itemsize > sizeof(array):
+ * tmp = PyMem_Malloc(self.view.itemsize)
+ * if tmp == NULL: # <<<<<<<<<<<<<<
+ * raise MemoryError
+ * item = tmp
+ */
+ __pyx_t_2 = ((__pyx_v_tmp == NULL) != 0);
+ if (unlikely(__pyx_t_2)) {
+
+ /* "View.MemoryView":461
+ * tmp = PyMem_Malloc(self.view.itemsize)
+ * if tmp == NULL:
+ * raise MemoryError # <<<<<<<<<<<<<<
+ * item = tmp
+ * else:
+ */
+ PyErr_NoMemory(); __PYX_ERR(1, 461, __pyx_L1_error)
+
+ /* "View.MemoryView":460
+ * if self.view.itemsize > sizeof(array):
+ * tmp = PyMem_Malloc(self.view.itemsize)
+ * if tmp == NULL: # <<<<<<<<<<<<<<
+ * raise MemoryError
+ * item = tmp
+ */
+ }
+
+ /* "View.MemoryView":462
+ * if tmp == NULL:
+ * raise MemoryError
+ * item = tmp # <<<<<<<<<<<<<<
+ * else:
+ * item = array
+ */
+ __pyx_v_item = __pyx_v_tmp;
+
+ /* "View.MemoryView":458
+ * dst_slice = get_slice_from_memview(dst, &tmp_slice)
+ *
+ * if self.view.itemsize > sizeof(array): # <<<<<<<<<<<<<<
+ * tmp = PyMem_Malloc(self.view.itemsize)
+ * if tmp == NULL:
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":464
+ * item = tmp
+ * else:
+ * item = array # <<<<<<<<<<<<<<
+ *
+ * try:
+ */
+ /*else*/ {
+ __pyx_v_item = ((void *)__pyx_v_array);
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":466
+ * item = array
+ *
+ * try: # <<<<<<<<<<<<<<
+ * if self.dtype_is_object:
+ * ( item)[0] = value
+ */
+ /*try:*/ {
+
+ /* "View.MemoryView":467
+ *
+ * try:
+ * if self.dtype_is_object: # <<<<<<<<<<<<<<
+ * ( item)[0] = value
+ * else:
+ */
+ __pyx_t_2 = (__pyx_v_self->dtype_is_object != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":468
+ * try:
+ * if self.dtype_is_object:
+ * ( item)[0] = value # <<<<<<<<<<<<<<
+ * else:
+ * self.assign_item_from_object( item, value)
+ */
+ (((PyObject **)__pyx_v_item)[0]) = ((PyObject *)__pyx_v_value);
+
+ /* "View.MemoryView":467
+ *
+ * try:
+ * if self.dtype_is_object: # <<<<<<<<<<<<<<
+ * ( item)[0] = value
+ * else:
+ */
+ goto __pyx_L8;
+ }
+
+ /* "View.MemoryView":470
+ * ( item)[0] = value
+ * else:
+ * self.assign_item_from_object( item, value) # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ /*else*/ {
+ __pyx_t_3 = ((struct __pyx_vtabstruct_memoryview *)__pyx_v_self->__pyx_vtab)->assign_item_from_object(__pyx_v_self, ((char *)__pyx_v_item), __pyx_v_value); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 470, __pyx_L6_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ }
+ __pyx_L8:;
+
+ /* "View.MemoryView":474
+ *
+ *
+ * if self.view.suboffsets != NULL: # <<<<<<<<<<<<<<
+ * assert_direct_dimensions(self.view.suboffsets, self.view.ndim)
+ * slice_assign_scalar(dst_slice, dst.view.ndim, self.view.itemsize,
+ */
+ __pyx_t_2 = ((__pyx_v_self->view.suboffsets != NULL) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":475
+ *
+ * if self.view.suboffsets != NULL:
+ * assert_direct_dimensions(self.view.suboffsets, self.view.ndim) # <<<<<<<<<<<<<<
+ * slice_assign_scalar(dst_slice, dst.view.ndim, self.view.itemsize,
+ * item, self.dtype_is_object)
+ */
+ __pyx_t_3 = assert_direct_dimensions(__pyx_v_self->view.suboffsets, __pyx_v_self->view.ndim); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 475, __pyx_L6_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+
+ /* "View.MemoryView":474
+ *
+ *
+ * if self.view.suboffsets != NULL: # <<<<<<<<<<<<<<
+ * assert_direct_dimensions(self.view.suboffsets, self.view.ndim)
+ * slice_assign_scalar(dst_slice, dst.view.ndim, self.view.itemsize,
+ */
+ }
+
+ /* "View.MemoryView":476
+ * if self.view.suboffsets != NULL:
+ * assert_direct_dimensions(self.view.suboffsets, self.view.ndim)
+ * slice_assign_scalar(dst_slice, dst.view.ndim, self.view.itemsize, # <<<<<<<<<<<<<<
+ * item, self.dtype_is_object)
+ * finally:
+ */
+ __pyx_memoryview_slice_assign_scalar(__pyx_v_dst_slice, __pyx_v_dst->view.ndim, __pyx_v_self->view.itemsize, __pyx_v_item, __pyx_v_self->dtype_is_object);
+ }
+
+ /* "View.MemoryView":479
+ * item, self.dtype_is_object)
+ * finally:
+ * PyMem_Free(tmp) # <<<<<<<<<<<<<<
+ *
+ * cdef setitem_indexed(self, index, value):
+ */
+ /*finally:*/ {
+ /*normal exit:*/{
+ PyMem_Free(__pyx_v_tmp);
+ goto __pyx_L7;
+ }
+ __pyx_L6_error:;
+ /*exception exit:*/{
+ __Pyx_PyThreadState_declare
+ __Pyx_PyThreadState_assign
+ __pyx_t_7 = 0; __pyx_t_8 = 0; __pyx_t_9 = 0; __pyx_t_10 = 0; __pyx_t_11 = 0; __pyx_t_12 = 0;
+ __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0;
+ if (PY_MAJOR_VERSION >= 3) __Pyx_ExceptionSwap(&__pyx_t_10, &__pyx_t_11, &__pyx_t_12);
+ if ((PY_MAJOR_VERSION < 3) || unlikely(__Pyx_GetException(&__pyx_t_7, &__pyx_t_8, &__pyx_t_9) < 0)) __Pyx_ErrFetch(&__pyx_t_7, &__pyx_t_8, &__pyx_t_9);
+ __Pyx_XGOTREF(__pyx_t_7);
+ __Pyx_XGOTREF(__pyx_t_8);
+ __Pyx_XGOTREF(__pyx_t_9);
+ __Pyx_XGOTREF(__pyx_t_10);
+ __Pyx_XGOTREF(__pyx_t_11);
+ __Pyx_XGOTREF(__pyx_t_12);
+ __pyx_t_4 = __pyx_lineno; __pyx_t_5 = __pyx_clineno; __pyx_t_6 = __pyx_filename;
+ {
+ PyMem_Free(__pyx_v_tmp);
+ }
+ if (PY_MAJOR_VERSION >= 3) {
+ __Pyx_XGIVEREF(__pyx_t_10);
+ __Pyx_XGIVEREF(__pyx_t_11);
+ __Pyx_XGIVEREF(__pyx_t_12);
+ __Pyx_ExceptionReset(__pyx_t_10, __pyx_t_11, __pyx_t_12);
+ }
+ __Pyx_XGIVEREF(__pyx_t_7);
+ __Pyx_XGIVEREF(__pyx_t_8);
+ __Pyx_XGIVEREF(__pyx_t_9);
+ __Pyx_ErrRestore(__pyx_t_7, __pyx_t_8, __pyx_t_9);
+ __pyx_t_7 = 0; __pyx_t_8 = 0; __pyx_t_9 = 0; __pyx_t_10 = 0; __pyx_t_11 = 0; __pyx_t_12 = 0;
+ __pyx_lineno = __pyx_t_4; __pyx_clineno = __pyx_t_5; __pyx_filename = __pyx_t_6;
+ goto __pyx_L1_error;
+ }
+ __pyx_L7:;
+ }
+
+ /* "View.MemoryView":449
+ * src.ndim, dst.ndim, self.dtype_is_object)
+ *
+ * cdef setitem_slice_assign_scalar(self, memoryview dst, value): # <<<<<<<<<<<<<<
+ * cdef int array[128]
+ * cdef void *tmp = NULL
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.setitem_slice_assign_scalar", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":481
+ * PyMem_Free(tmp)
+ *
+ * cdef setitem_indexed(self, index, value): # <<<<<<<<<<<<<<
+ * cdef char *itemp = self.get_item_pointer(index)
+ * self.assign_item_from_object(itemp, value)
+ */
+
+static PyObject *__pyx_memoryview_setitem_indexed(struct __pyx_memoryview_obj *__pyx_v_self, PyObject *__pyx_v_index, PyObject *__pyx_v_value) {
+ char *__pyx_v_itemp;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ char *__pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("setitem_indexed", 0);
+
+ /* "View.MemoryView":482
+ *
+ * cdef setitem_indexed(self, index, value):
+ * cdef char *itemp = self.get_item_pointer(index) # <<<<<<<<<<<<<<
+ * self.assign_item_from_object(itemp, value)
+ *
+ */
+ __pyx_t_1 = ((struct __pyx_vtabstruct_memoryview *)__pyx_v_self->__pyx_vtab)->get_item_pointer(__pyx_v_self, __pyx_v_index); if (unlikely(__pyx_t_1 == ((char *)NULL))) __PYX_ERR(1, 482, __pyx_L1_error)
+ __pyx_v_itemp = __pyx_t_1;
+
+ /* "View.MemoryView":483
+ * cdef setitem_indexed(self, index, value):
+ * cdef char *itemp = self.get_item_pointer(index)
+ * self.assign_item_from_object(itemp, value) # <<<<<<<<<<<<<<
+ *
+ * cdef convert_item_to_object(self, char *itemp):
+ */
+ __pyx_t_2 = ((struct __pyx_vtabstruct_memoryview *)__pyx_v_self->__pyx_vtab)->assign_item_from_object(__pyx_v_self, __pyx_v_itemp, __pyx_v_value); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 483, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+
+ /* "View.MemoryView":481
+ * PyMem_Free(tmp)
+ *
+ * cdef setitem_indexed(self, index, value): # <<<<<<<<<<<<<<
+ * cdef char *itemp = self.get_item_pointer(index)
+ * self.assign_item_from_object(itemp, value)
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.setitem_indexed", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":485
+ * self.assign_item_from_object(itemp, value)
+ *
+ * cdef convert_item_to_object(self, char *itemp): # <<<<<<<<<<<<<<
+ * """Only used if instantiated manually by the user, or if Cython doesn't
+ * know how to convert the type"""
+ */
+
+static PyObject *__pyx_memoryview_convert_item_to_object(struct __pyx_memoryview_obj *__pyx_v_self, char *__pyx_v_itemp) {
+ PyObject *__pyx_v_struct = NULL;
+ PyObject *__pyx_v_bytesitem = 0;
+ PyObject *__pyx_v_result = NULL;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ PyObject *__pyx_t_4 = NULL;
+ PyObject *__pyx_t_5 = NULL;
+ PyObject *__pyx_t_6 = NULL;
+ PyObject *__pyx_t_7 = NULL;
+ int __pyx_t_8;
+ PyObject *__pyx_t_9 = NULL;
+ size_t __pyx_t_10;
+ int __pyx_t_11;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("convert_item_to_object", 0);
+
+ /* "View.MemoryView":488
+ * """Only used if instantiated manually by the user, or if Cython doesn't
+ * know how to convert the type"""
+ * import struct # <<<<<<<<<<<<<<
+ * cdef bytes bytesitem
+ *
+ */
+ __pyx_t_1 = __Pyx_Import(__pyx_n_s_struct, 0, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 488, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_v_struct = __pyx_t_1;
+ __pyx_t_1 = 0;
+
+ /* "View.MemoryView":491
+ * cdef bytes bytesitem
+ *
+ * bytesitem = itemp[:self.view.itemsize] # <<<<<<<<<<<<<<
+ * try:
+ * result = struct.unpack(self.view.format, bytesitem)
+ */
+ __pyx_t_1 = __Pyx_PyBytes_FromStringAndSize(__pyx_v_itemp + 0, __pyx_v_self->view.itemsize - 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 491, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_v_bytesitem = ((PyObject*)__pyx_t_1);
+ __pyx_t_1 = 0;
+
+ /* "View.MemoryView":492
+ *
+ * bytesitem = itemp[:self.view.itemsize]
+ * try: # <<<<<<<<<<<<<<
+ * result = struct.unpack(self.view.format, bytesitem)
+ * except struct.error:
+ */
+ {
+ __Pyx_PyThreadState_declare
+ __Pyx_PyThreadState_assign
+ __Pyx_ExceptionSave(&__pyx_t_2, &__pyx_t_3, &__pyx_t_4);
+ __Pyx_XGOTREF(__pyx_t_2);
+ __Pyx_XGOTREF(__pyx_t_3);
+ __Pyx_XGOTREF(__pyx_t_4);
+ /*try:*/ {
+
+ /* "View.MemoryView":493
+ * bytesitem = itemp[:self.view.itemsize]
+ * try:
+ * result = struct.unpack(self.view.format, bytesitem) # <<<<<<<<<<<<<<
+ * except struct.error:
+ * raise ValueError("Unable to convert item to object")
+ */
+ __pyx_t_5 = __Pyx_PyObject_GetAttrStr(__pyx_v_struct, __pyx_n_s_unpack); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 493, __pyx_L3_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __pyx_t_6 = __Pyx_PyBytes_FromString(__pyx_v_self->view.format); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 493, __pyx_L3_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ __pyx_t_7 = NULL;
+ __pyx_t_8 = 0;
+ if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_5))) {
+ __pyx_t_7 = PyMethod_GET_SELF(__pyx_t_5);
+ if (likely(__pyx_t_7)) {
+ PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_5);
+ __Pyx_INCREF(__pyx_t_7);
+ __Pyx_INCREF(function);
+ __Pyx_DECREF_SET(__pyx_t_5, function);
+ __pyx_t_8 = 1;
+ }
+ }
+ #if CYTHON_FAST_PYCALL
+ if (PyFunction_Check(__pyx_t_5)) {
+ PyObject *__pyx_temp[3] = {__pyx_t_7, __pyx_t_6, __pyx_v_bytesitem};
+ __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_5, __pyx_temp+1-__pyx_t_8, 2+__pyx_t_8); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 493, __pyx_L3_error)
+ __Pyx_XDECREF(__pyx_t_7); __pyx_t_7 = 0;
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
+ } else
+ #endif
+ #if CYTHON_FAST_PYCCALL
+ if (__Pyx_PyFastCFunction_Check(__pyx_t_5)) {
+ PyObject *__pyx_temp[3] = {__pyx_t_7, __pyx_t_6, __pyx_v_bytesitem};
+ __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_5, __pyx_temp+1-__pyx_t_8, 2+__pyx_t_8); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 493, __pyx_L3_error)
+ __Pyx_XDECREF(__pyx_t_7); __pyx_t_7 = 0;
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
+ } else
+ #endif
+ {
+ __pyx_t_9 = PyTuple_New(2+__pyx_t_8); if (unlikely(!__pyx_t_9)) __PYX_ERR(1, 493, __pyx_L3_error)
+ __Pyx_GOTREF(__pyx_t_9);
+ if (__pyx_t_7) {
+ __Pyx_GIVEREF(__pyx_t_7); PyTuple_SET_ITEM(__pyx_t_9, 0, __pyx_t_7); __pyx_t_7 = NULL;
+ }
+ __Pyx_GIVEREF(__pyx_t_6);
+ PyTuple_SET_ITEM(__pyx_t_9, 0+__pyx_t_8, __pyx_t_6);
+ __Pyx_INCREF(__pyx_v_bytesitem);
+ __Pyx_GIVEREF(__pyx_v_bytesitem);
+ PyTuple_SET_ITEM(__pyx_t_9, 1+__pyx_t_8, __pyx_v_bytesitem);
+ __pyx_t_6 = 0;
+ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_5, __pyx_t_9, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 493, __pyx_L3_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
+ }
+ __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
+ __pyx_v_result = __pyx_t_1;
+ __pyx_t_1 = 0;
+
+ /* "View.MemoryView":492
+ *
+ * bytesitem = itemp[:self.view.itemsize]
+ * try: # <<<<<<<<<<<<<<
+ * result = struct.unpack(self.view.format, bytesitem)
+ * except struct.error:
+ */
+ }
+
+ /* "View.MemoryView":497
+ * raise ValueError("Unable to convert item to object")
+ * else:
+ * if len(self.view.format) == 1: # <<<<<<<<<<<<<<
+ * return result[0]
+ * return result
+ */
+ /*else:*/ {
+ __pyx_t_10 = strlen(__pyx_v_self->view.format);
+ __pyx_t_11 = ((__pyx_t_10 == 1) != 0);
+ if (__pyx_t_11) {
+
+ /* "View.MemoryView":498
+ * else:
+ * if len(self.view.format) == 1:
+ * return result[0] # <<<<<<<<<<<<<<
+ * return result
+ *
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_GetItemInt(__pyx_v_result, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 498, __pyx_L5_except_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L6_except_return;
+
+ /* "View.MemoryView":497
+ * raise ValueError("Unable to convert item to object")
+ * else:
+ * if len(self.view.format) == 1: # <<<<<<<<<<<<<<
+ * return result[0]
+ * return result
+ */
+ }
+
+ /* "View.MemoryView":499
+ * if len(self.view.format) == 1:
+ * return result[0]
+ * return result # <<<<<<<<<<<<<<
+ *
+ * cdef assign_item_from_object(self, char *itemp, object value):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(__pyx_v_result);
+ __pyx_r = __pyx_v_result;
+ goto __pyx_L6_except_return;
+ }
+ __pyx_L3_error:;
+ __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0;
+ __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0;
+ __Pyx_XDECREF(__pyx_t_7); __pyx_t_7 = 0;
+ __Pyx_XDECREF(__pyx_t_9); __pyx_t_9 = 0;
+
+ /* "View.MemoryView":494
+ * try:
+ * result = struct.unpack(self.view.format, bytesitem)
+ * except struct.error: # <<<<<<<<<<<<<<
+ * raise ValueError("Unable to convert item to object")
+ * else:
+ */
+ __Pyx_ErrFetch(&__pyx_t_1, &__pyx_t_5, &__pyx_t_9);
+ __pyx_t_6 = __Pyx_PyObject_GetAttrStr(__pyx_v_struct, __pyx_n_s_error); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 494, __pyx_L5_except_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ __pyx_t_8 = __Pyx_PyErr_GivenExceptionMatches(__pyx_t_1, __pyx_t_6);
+ __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
+ __Pyx_ErrRestore(__pyx_t_1, __pyx_t_5, __pyx_t_9);
+ __pyx_t_1 = 0; __pyx_t_5 = 0; __pyx_t_9 = 0;
+ if (__pyx_t_8) {
+ __Pyx_AddTraceback("View.MemoryView.memoryview.convert_item_to_object", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ if (__Pyx_GetException(&__pyx_t_9, &__pyx_t_5, &__pyx_t_1) < 0) __PYX_ERR(1, 494, __pyx_L5_except_error)
+ __Pyx_GOTREF(__pyx_t_9);
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_GOTREF(__pyx_t_1);
+
+ /* "View.MemoryView":495
+ * result = struct.unpack(self.view.format, bytesitem)
+ * except struct.error:
+ * raise ValueError("Unable to convert item to object") # <<<<<<<<<<<<<<
+ * else:
+ * if len(self.view.format) == 1:
+ */
+ __pyx_t_6 = __Pyx_PyObject_Call(__pyx_builtin_ValueError, __pyx_tuple__13, NULL); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 495, __pyx_L5_except_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ __Pyx_Raise(__pyx_t_6, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
+ __PYX_ERR(1, 495, __pyx_L5_except_error)
+ }
+ goto __pyx_L5_except_error;
+ __pyx_L5_except_error:;
+
+ /* "View.MemoryView":492
+ *
+ * bytesitem = itemp[:self.view.itemsize]
+ * try: # <<<<<<<<<<<<<<
+ * result = struct.unpack(self.view.format, bytesitem)
+ * except struct.error:
+ */
+ __Pyx_XGIVEREF(__pyx_t_2);
+ __Pyx_XGIVEREF(__pyx_t_3);
+ __Pyx_XGIVEREF(__pyx_t_4);
+ __Pyx_ExceptionReset(__pyx_t_2, __pyx_t_3, __pyx_t_4);
+ goto __pyx_L1_error;
+ __pyx_L6_except_return:;
+ __Pyx_XGIVEREF(__pyx_t_2);
+ __Pyx_XGIVEREF(__pyx_t_3);
+ __Pyx_XGIVEREF(__pyx_t_4);
+ __Pyx_ExceptionReset(__pyx_t_2, __pyx_t_3, __pyx_t_4);
+ goto __pyx_L0;
+ }
+
+ /* "View.MemoryView":485
+ * self.assign_item_from_object(itemp, value)
+ *
+ * cdef convert_item_to_object(self, char *itemp): # <<<<<<<<<<<<<<
+ * """Only used if instantiated manually by the user, or if Cython doesn't
+ * know how to convert the type"""
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_XDECREF(__pyx_t_6);
+ __Pyx_XDECREF(__pyx_t_7);
+ __Pyx_XDECREF(__pyx_t_9);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.convert_item_to_object", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_struct);
+ __Pyx_XDECREF(__pyx_v_bytesitem);
+ __Pyx_XDECREF(__pyx_v_result);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":501
+ * return result
+ *
+ * cdef assign_item_from_object(self, char *itemp, object value): # <<<<<<<<<<<<<<
+ * """Only used if instantiated manually by the user, or if Cython doesn't
+ * know how to convert the type"""
+ */
+
+static PyObject *__pyx_memoryview_assign_item_from_object(struct __pyx_memoryview_obj *__pyx_v_self, char *__pyx_v_itemp, PyObject *__pyx_v_value) {
+ PyObject *__pyx_v_struct = NULL;
+ char __pyx_v_c;
+ PyObject *__pyx_v_bytesvalue = 0;
+ Py_ssize_t __pyx_v_i;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_t_2;
+ int __pyx_t_3;
+ PyObject *__pyx_t_4 = NULL;
+ PyObject *__pyx_t_5 = NULL;
+ PyObject *__pyx_t_6 = NULL;
+ int __pyx_t_7;
+ PyObject *__pyx_t_8 = NULL;
+ Py_ssize_t __pyx_t_9;
+ PyObject *__pyx_t_10 = NULL;
+ char *__pyx_t_11;
+ char *__pyx_t_12;
+ char *__pyx_t_13;
+ char *__pyx_t_14;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("assign_item_from_object", 0);
+
+ /* "View.MemoryView":504
+ * """Only used if instantiated manually by the user, or if Cython doesn't
+ * know how to convert the type"""
+ * import struct # <<<<<<<<<<<<<<
+ * cdef char c
+ * cdef bytes bytesvalue
+ */
+ __pyx_t_1 = __Pyx_Import(__pyx_n_s_struct, 0, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 504, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_v_struct = __pyx_t_1;
+ __pyx_t_1 = 0;
+
+ /* "View.MemoryView":509
+ * cdef Py_ssize_t i
+ *
+ * if isinstance(value, tuple): # <<<<<<<<<<<<<<
+ * bytesvalue = struct.pack(self.view.format, *value)
+ * else:
+ */
+ __pyx_t_2 = PyTuple_Check(__pyx_v_value);
+ __pyx_t_3 = (__pyx_t_2 != 0);
+ if (__pyx_t_3) {
+
+ /* "View.MemoryView":510
+ *
+ * if isinstance(value, tuple):
+ * bytesvalue = struct.pack(self.view.format, *value) # <<<<<<<<<<<<<<
+ * else:
+ * bytesvalue = struct.pack(self.view.format, value)
+ */
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_v_struct, __pyx_n_s_pack); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 510, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_4 = __Pyx_PyBytes_FromString(__pyx_v_self->view.format); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 510, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_5 = PyTuple_New(1); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 510, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_GIVEREF(__pyx_t_4);
+ PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_t_4);
+ __pyx_t_4 = 0;
+ __pyx_t_4 = __Pyx_PySequence_Tuple(__pyx_v_value); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 510, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_6 = PyNumber_Add(__pyx_t_5, __pyx_t_4); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 510, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+ __pyx_t_4 = __Pyx_PyObject_Call(__pyx_t_1, __pyx_t_6, NULL); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 510, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
+ if (!(likely(PyBytes_CheckExact(__pyx_t_4))||((__pyx_t_4) == Py_None)||(PyErr_Format(PyExc_TypeError, "Expected %.16s, got %.200s", "bytes", Py_TYPE(__pyx_t_4)->tp_name), 0))) __PYX_ERR(1, 510, __pyx_L1_error)
+ __pyx_v_bytesvalue = ((PyObject*)__pyx_t_4);
+ __pyx_t_4 = 0;
+
+ /* "View.MemoryView":509
+ * cdef Py_ssize_t i
+ *
+ * if isinstance(value, tuple): # <<<<<<<<<<<<<<
+ * bytesvalue = struct.pack(self.view.format, *value)
+ * else:
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":512
+ * bytesvalue = struct.pack(self.view.format, *value)
+ * else:
+ * bytesvalue = struct.pack(self.view.format, value) # <<<<<<<<<<<<<<
+ *
+ * for i, c in enumerate(bytesvalue):
+ */
+ /*else*/ {
+ __pyx_t_6 = __Pyx_PyObject_GetAttrStr(__pyx_v_struct, __pyx_n_s_pack); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 512, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ __pyx_t_1 = __Pyx_PyBytes_FromString(__pyx_v_self->view.format); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 512, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_5 = NULL;
+ __pyx_t_7 = 0;
+ if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_6))) {
+ __pyx_t_5 = PyMethod_GET_SELF(__pyx_t_6);
+ if (likely(__pyx_t_5)) {
+ PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_6);
+ __Pyx_INCREF(__pyx_t_5);
+ __Pyx_INCREF(function);
+ __Pyx_DECREF_SET(__pyx_t_6, function);
+ __pyx_t_7 = 1;
+ }
+ }
+ #if CYTHON_FAST_PYCALL
+ if (PyFunction_Check(__pyx_t_6)) {
+ PyObject *__pyx_temp[3] = {__pyx_t_5, __pyx_t_1, __pyx_v_value};
+ __pyx_t_4 = __Pyx_PyFunction_FastCall(__pyx_t_6, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 512, __pyx_L1_error)
+ __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0;
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ } else
+ #endif
+ #if CYTHON_FAST_PYCCALL
+ if (__Pyx_PyFastCFunction_Check(__pyx_t_6)) {
+ PyObject *__pyx_temp[3] = {__pyx_t_5, __pyx_t_1, __pyx_v_value};
+ __pyx_t_4 = __Pyx_PyCFunction_FastCall(__pyx_t_6, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 512, __pyx_L1_error)
+ __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0;
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ } else
+ #endif
+ {
+ __pyx_t_8 = PyTuple_New(2+__pyx_t_7); if (unlikely(!__pyx_t_8)) __PYX_ERR(1, 512, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_8);
+ if (__pyx_t_5) {
+ __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_8, 0, __pyx_t_5); __pyx_t_5 = NULL;
+ }
+ __Pyx_GIVEREF(__pyx_t_1);
+ PyTuple_SET_ITEM(__pyx_t_8, 0+__pyx_t_7, __pyx_t_1);
+ __Pyx_INCREF(__pyx_v_value);
+ __Pyx_GIVEREF(__pyx_v_value);
+ PyTuple_SET_ITEM(__pyx_t_8, 1+__pyx_t_7, __pyx_v_value);
+ __pyx_t_1 = 0;
+ __pyx_t_4 = __Pyx_PyObject_Call(__pyx_t_6, __pyx_t_8, NULL); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 512, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
+ }
+ __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
+ if (!(likely(PyBytes_CheckExact(__pyx_t_4))||((__pyx_t_4) == Py_None)||(PyErr_Format(PyExc_TypeError, "Expected %.16s, got %.200s", "bytes", Py_TYPE(__pyx_t_4)->tp_name), 0))) __PYX_ERR(1, 512, __pyx_L1_error)
+ __pyx_v_bytesvalue = ((PyObject*)__pyx_t_4);
+ __pyx_t_4 = 0;
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":514
+ * bytesvalue = struct.pack(self.view.format, value)
+ *
+ * for i, c in enumerate(bytesvalue): # <<<<<<<<<<<<<<
+ * itemp[i] = c
+ *
+ */
+ __pyx_t_9 = 0;
+ if (unlikely(__pyx_v_bytesvalue == Py_None)) {
+ PyErr_SetString(PyExc_TypeError, "'NoneType' is not iterable");
+ __PYX_ERR(1, 514, __pyx_L1_error)
+ }
+ __Pyx_INCREF(__pyx_v_bytesvalue);
+ __pyx_t_10 = __pyx_v_bytesvalue;
+ __pyx_t_12 = PyBytes_AS_STRING(__pyx_t_10);
+ __pyx_t_13 = (__pyx_t_12 + PyBytes_GET_SIZE(__pyx_t_10));
+ for (__pyx_t_14 = __pyx_t_12; __pyx_t_14 < __pyx_t_13; __pyx_t_14++) {
+ __pyx_t_11 = __pyx_t_14;
+ __pyx_v_c = (__pyx_t_11[0]);
+
+ /* "View.MemoryView":515
+ *
+ * for i, c in enumerate(bytesvalue):
+ * itemp[i] = c # <<<<<<<<<<<<<<
+ *
+ * @cname('getbuffer')
+ */
+ __pyx_v_i = __pyx_t_9;
+
+ /* "View.MemoryView":514
+ * bytesvalue = struct.pack(self.view.format, value)
+ *
+ * for i, c in enumerate(bytesvalue): # <<<<<<<<<<<<<<
+ * itemp[i] = c
+ *
+ */
+ __pyx_t_9 = (__pyx_t_9 + 1);
+
+ /* "View.MemoryView":515
+ *
+ * for i, c in enumerate(bytesvalue):
+ * itemp[i] = c # <<<<<<<<<<<<<<
+ *
+ * @cname('getbuffer')
+ */
+ (__pyx_v_itemp[__pyx_v_i]) = __pyx_v_c;
+ }
+ __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0;
+
+ /* "View.MemoryView":501
+ * return result
+ *
+ * cdef assign_item_from_object(self, char *itemp, object value): # <<<<<<<<<<<<<<
+ * """Only used if instantiated manually by the user, or if Cython doesn't
+ * know how to convert the type"""
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_XDECREF(__pyx_t_6);
+ __Pyx_XDECREF(__pyx_t_8);
+ __Pyx_XDECREF(__pyx_t_10);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.assign_item_from_object", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_struct);
+ __Pyx_XDECREF(__pyx_v_bytesvalue);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":518
+ *
+ * @cname('getbuffer')
+ * def __getbuffer__(self, Py_buffer *info, int flags): # <<<<<<<<<<<<<<
+ * if flags & PyBUF_WRITABLE and self.view.readonly:
+ * raise ValueError("Cannot create writable memory view from read-only memoryview")
+ */
+
+/* Python wrapper */
+static CYTHON_UNUSED int __pyx_memoryview_getbuffer(PyObject *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags); /*proto*/
+static CYTHON_UNUSED int __pyx_memoryview_getbuffer(PyObject *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__getbuffer__ (wrapper)", 0);
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_8__getbuffer__(((struct __pyx_memoryview_obj *)__pyx_v_self), ((Py_buffer *)__pyx_v_info), ((int)__pyx_v_flags));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static int __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_8__getbuffer__(struct __pyx_memoryview_obj *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ Py_ssize_t *__pyx_t_4;
+ char *__pyx_t_5;
+ void *__pyx_t_6;
+ int __pyx_t_7;
+ Py_ssize_t __pyx_t_8;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ if (__pyx_v_info == NULL) {
+ PyErr_SetString(PyExc_BufferError, "PyObject_GetBuffer: view==NULL argument is obsolete");
+ return -1;
+ }
+ __Pyx_RefNannySetupContext("__getbuffer__", 0);
+ __pyx_v_info->obj = Py_None; __Pyx_INCREF(Py_None);
+ __Pyx_GIVEREF(__pyx_v_info->obj);
+
+ /* "View.MemoryView":519
+ * @cname('getbuffer')
+ * def __getbuffer__(self, Py_buffer *info, int flags):
+ * if flags & PyBUF_WRITABLE and self.view.readonly: # <<<<<<<<<<<<<<
+ * raise ValueError("Cannot create writable memory view from read-only memoryview")
+ *
+ */
+ __pyx_t_2 = ((__pyx_v_flags & PyBUF_WRITABLE) != 0);
+ if (__pyx_t_2) {
+ } else {
+ __pyx_t_1 = __pyx_t_2;
+ goto __pyx_L4_bool_binop_done;
+ }
+ __pyx_t_2 = (__pyx_v_self->view.readonly != 0);
+ __pyx_t_1 = __pyx_t_2;
+ __pyx_L4_bool_binop_done:;
+ if (unlikely(__pyx_t_1)) {
+
+ /* "View.MemoryView":520
+ * def __getbuffer__(self, Py_buffer *info, int flags):
+ * if flags & PyBUF_WRITABLE and self.view.readonly:
+ * raise ValueError("Cannot create writable memory view from read-only memoryview") # <<<<<<<<<<<<<<
+ *
+ * if flags & PyBUF_ND:
+ */
+ __pyx_t_3 = __Pyx_PyObject_Call(__pyx_builtin_ValueError, __pyx_tuple__14, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 520, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_Raise(__pyx_t_3, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __PYX_ERR(1, 520, __pyx_L1_error)
+
+ /* "View.MemoryView":519
+ * @cname('getbuffer')
+ * def __getbuffer__(self, Py_buffer *info, int flags):
+ * if flags & PyBUF_WRITABLE and self.view.readonly: # <<<<<<<<<<<<<<
+ * raise ValueError("Cannot create writable memory view from read-only memoryview")
+ *
+ */
+ }
+
+ /* "View.MemoryView":522
+ * raise ValueError("Cannot create writable memory view from read-only memoryview")
+ *
+ * if flags & PyBUF_ND: # <<<<<<<<<<<<<<
+ * info.shape = self.view.shape
+ * else:
+ */
+ __pyx_t_1 = ((__pyx_v_flags & PyBUF_ND) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":523
+ *
+ * if flags & PyBUF_ND:
+ * info.shape = self.view.shape # <<<<<<<<<<<<<<
+ * else:
+ * info.shape = NULL
+ */
+ __pyx_t_4 = __pyx_v_self->view.shape;
+ __pyx_v_info->shape = __pyx_t_4;
+
+ /* "View.MemoryView":522
+ * raise ValueError("Cannot create writable memory view from read-only memoryview")
+ *
+ * if flags & PyBUF_ND: # <<<<<<<<<<<<<<
+ * info.shape = self.view.shape
+ * else:
+ */
+ goto __pyx_L6;
+ }
+
+ /* "View.MemoryView":525
+ * info.shape = self.view.shape
+ * else:
+ * info.shape = NULL # <<<<<<<<<<<<<<
+ *
+ * if flags & PyBUF_STRIDES:
+ */
+ /*else*/ {
+ __pyx_v_info->shape = NULL;
+ }
+ __pyx_L6:;
+
+ /* "View.MemoryView":527
+ * info.shape = NULL
+ *
+ * if flags & PyBUF_STRIDES: # <<<<<<<<<<<<<<
+ * info.strides = self.view.strides
+ * else:
+ */
+ __pyx_t_1 = ((__pyx_v_flags & PyBUF_STRIDES) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":528
+ *
+ * if flags & PyBUF_STRIDES:
+ * info.strides = self.view.strides # <<<<<<<<<<<<<<
+ * else:
+ * info.strides = NULL
+ */
+ __pyx_t_4 = __pyx_v_self->view.strides;
+ __pyx_v_info->strides = __pyx_t_4;
+
+ /* "View.MemoryView":527
+ * info.shape = NULL
+ *
+ * if flags & PyBUF_STRIDES: # <<<<<<<<<<<<<<
+ * info.strides = self.view.strides
+ * else:
+ */
+ goto __pyx_L7;
+ }
+
+ /* "View.MemoryView":530
+ * info.strides = self.view.strides
+ * else:
+ * info.strides = NULL # <<<<<<<<<<<<<<
+ *
+ * if flags & PyBUF_INDIRECT:
+ */
+ /*else*/ {
+ __pyx_v_info->strides = NULL;
+ }
+ __pyx_L7:;
+
+ /* "View.MemoryView":532
+ * info.strides = NULL
+ *
+ * if flags & PyBUF_INDIRECT: # <<<<<<<<<<<<<<
+ * info.suboffsets = self.view.suboffsets
+ * else:
+ */
+ __pyx_t_1 = ((__pyx_v_flags & PyBUF_INDIRECT) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":533
+ *
+ * if flags & PyBUF_INDIRECT:
+ * info.suboffsets = self.view.suboffsets # <<<<<<<<<<<<<<
+ * else:
+ * info.suboffsets = NULL
+ */
+ __pyx_t_4 = __pyx_v_self->view.suboffsets;
+ __pyx_v_info->suboffsets = __pyx_t_4;
+
+ /* "View.MemoryView":532
+ * info.strides = NULL
+ *
+ * if flags & PyBUF_INDIRECT: # <<<<<<<<<<<<<<
+ * info.suboffsets = self.view.suboffsets
+ * else:
+ */
+ goto __pyx_L8;
+ }
+
+ /* "View.MemoryView":535
+ * info.suboffsets = self.view.suboffsets
+ * else:
+ * info.suboffsets = NULL # <<<<<<<<<<<<<<
+ *
+ * if flags & PyBUF_FORMAT:
+ */
+ /*else*/ {
+ __pyx_v_info->suboffsets = NULL;
+ }
+ __pyx_L8:;
+
+ /* "View.MemoryView":537
+ * info.suboffsets = NULL
+ *
+ * if flags & PyBUF_FORMAT: # <<<<<<<<<<<<<<
+ * info.format = self.view.format
+ * else:
+ */
+ __pyx_t_1 = ((__pyx_v_flags & PyBUF_FORMAT) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":538
+ *
+ * if flags & PyBUF_FORMAT:
+ * info.format = self.view.format # <<<<<<<<<<<<<<
+ * else:
+ * info.format = NULL
+ */
+ __pyx_t_5 = __pyx_v_self->view.format;
+ __pyx_v_info->format = __pyx_t_5;
+
+ /* "View.MemoryView":537
+ * info.suboffsets = NULL
+ *
+ * if flags & PyBUF_FORMAT: # <<<<<<<<<<<<<<
+ * info.format = self.view.format
+ * else:
+ */
+ goto __pyx_L9;
+ }
+
+ /* "View.MemoryView":540
+ * info.format = self.view.format
+ * else:
+ * info.format = NULL # <<<<<<<<<<<<<<
+ *
+ * info.buf = self.view.buf
+ */
+ /*else*/ {
+ __pyx_v_info->format = NULL;
+ }
+ __pyx_L9:;
+
+ /* "View.MemoryView":542
+ * info.format = NULL
+ *
+ * info.buf = self.view.buf # <<<<<<<<<<<<<<
+ * info.ndim = self.view.ndim
+ * info.itemsize = self.view.itemsize
+ */
+ __pyx_t_6 = __pyx_v_self->view.buf;
+ __pyx_v_info->buf = __pyx_t_6;
+
+ /* "View.MemoryView":543
+ *
+ * info.buf = self.view.buf
+ * info.ndim = self.view.ndim # <<<<<<<<<<<<<<
+ * info.itemsize = self.view.itemsize
+ * info.len = self.view.len
+ */
+ __pyx_t_7 = __pyx_v_self->view.ndim;
+ __pyx_v_info->ndim = __pyx_t_7;
+
+ /* "View.MemoryView":544
+ * info.buf = self.view.buf
+ * info.ndim = self.view.ndim
+ * info.itemsize = self.view.itemsize # <<<<<<<<<<<<<<
+ * info.len = self.view.len
+ * info.readonly = self.view.readonly
+ */
+ __pyx_t_8 = __pyx_v_self->view.itemsize;
+ __pyx_v_info->itemsize = __pyx_t_8;
+
+ /* "View.MemoryView":545
+ * info.ndim = self.view.ndim
+ * info.itemsize = self.view.itemsize
+ * info.len = self.view.len # <<<<<<<<<<<<<<
+ * info.readonly = self.view.readonly
+ * info.obj = self
+ */
+ __pyx_t_8 = __pyx_v_self->view.len;
+ __pyx_v_info->len = __pyx_t_8;
+
+ /* "View.MemoryView":546
+ * info.itemsize = self.view.itemsize
+ * info.len = self.view.len
+ * info.readonly = self.view.readonly # <<<<<<<<<<<<<<
+ * info.obj = self
+ *
+ */
+ __pyx_t_1 = __pyx_v_self->view.readonly;
+ __pyx_v_info->readonly = __pyx_t_1;
+
+ /* "View.MemoryView":547
+ * info.len = self.view.len
+ * info.readonly = self.view.readonly
+ * info.obj = self # <<<<<<<<<<<<<<
+ *
+ * __pyx_getbuffer = capsule( &__pyx_memoryview_getbuffer, "getbuffer(obj, view, flags)")
+ */
+ __Pyx_INCREF(((PyObject *)__pyx_v_self));
+ __Pyx_GIVEREF(((PyObject *)__pyx_v_self));
+ __Pyx_GOTREF(__pyx_v_info->obj);
+ __Pyx_DECREF(__pyx_v_info->obj);
+ __pyx_v_info->obj = ((PyObject *)__pyx_v_self);
+
+ /* "View.MemoryView":518
+ *
+ * @cname('getbuffer')
+ * def __getbuffer__(self, Py_buffer *info, int flags): # <<<<<<<<<<<<<<
+ * if flags & PyBUF_WRITABLE and self.view.readonly:
+ * raise ValueError("Cannot create writable memory view from read-only memoryview")
+ */
+
+ /* function exit code */
+ __pyx_r = 0;
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.__getbuffer__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = -1;
+ if (__pyx_v_info->obj != NULL) {
+ __Pyx_GOTREF(__pyx_v_info->obj);
+ __Pyx_DECREF(__pyx_v_info->obj); __pyx_v_info->obj = 0;
+ }
+ goto __pyx_L2;
+ __pyx_L0:;
+ if (__pyx_v_info->obj == Py_None) {
+ __Pyx_GOTREF(__pyx_v_info->obj);
+ __Pyx_DECREF(__pyx_v_info->obj); __pyx_v_info->obj = 0;
+ }
+ __pyx_L2:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":553
+ *
+ * @property
+ * def T(self): # <<<<<<<<<<<<<<
+ * cdef _memoryviewslice result = memoryview_copy(self)
+ * transpose_memslice(&result.from_slice)
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_1T_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_1T_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_10memoryview_1T___get__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_1T___get__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ struct __pyx_memoryviewslice_obj *__pyx_v_result = 0;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_t_2;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":554
+ * @property
+ * def T(self):
+ * cdef _memoryviewslice result = memoryview_copy(self) # <<<<<<<<<<<<<<
+ * transpose_memslice(&result.from_slice)
+ * return result
+ */
+ __pyx_t_1 = __pyx_memoryview_copy_object(__pyx_v_self); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 554, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ if (!(likely(((__pyx_t_1) == Py_None) || likely(__Pyx_TypeTest(__pyx_t_1, __pyx_memoryviewslice_type))))) __PYX_ERR(1, 554, __pyx_L1_error)
+ __pyx_v_result = ((struct __pyx_memoryviewslice_obj *)__pyx_t_1);
+ __pyx_t_1 = 0;
+
+ /* "View.MemoryView":555
+ * def T(self):
+ * cdef _memoryviewslice result = memoryview_copy(self)
+ * transpose_memslice(&result.from_slice) # <<<<<<<<<<<<<<
+ * return result
+ *
+ */
+ __pyx_t_2 = __pyx_memslice_transpose((&__pyx_v_result->from_slice)); if (unlikely(__pyx_t_2 == ((int)0))) __PYX_ERR(1, 555, __pyx_L1_error)
+
+ /* "View.MemoryView":556
+ * cdef _memoryviewslice result = memoryview_copy(self)
+ * transpose_memslice(&result.from_slice)
+ * return result # <<<<<<<<<<<<<<
+ *
+ * @property
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(((PyObject *)__pyx_v_result));
+ __pyx_r = ((PyObject *)__pyx_v_result);
+ goto __pyx_L0;
+
+ /* "View.MemoryView":553
+ *
+ * @property
+ * def T(self): # <<<<<<<<<<<<<<
+ * cdef _memoryviewslice result = memoryview_copy(self)
+ * transpose_memslice(&result.from_slice)
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.T.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XDECREF((PyObject *)__pyx_v_result);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":559
+ *
+ * @property
+ * def base(self): # <<<<<<<<<<<<<<
+ * return self.obj
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_4base_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_4base_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_10memoryview_4base___get__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_4base___get__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":560
+ * @property
+ * def base(self):
+ * return self.obj # <<<<<<<<<<<<<<
+ *
+ * @property
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(__pyx_v_self->obj);
+ __pyx_r = __pyx_v_self->obj;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":559
+ *
+ * @property
+ * def base(self): # <<<<<<<<<<<<<<
+ * return self.obj
+ *
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":563
+ *
+ * @property
+ * def shape(self): # <<<<<<<<<<<<<<
+ * return tuple([length for length in self.view.shape[:self.view.ndim]])
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_5shape_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_5shape_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_10memoryview_5shape___get__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_5shape___get__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ Py_ssize_t __pyx_v_length;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ Py_ssize_t *__pyx_t_2;
+ Py_ssize_t *__pyx_t_3;
+ Py_ssize_t *__pyx_t_4;
+ PyObject *__pyx_t_5 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":564
+ * @property
+ * def shape(self):
+ * return tuple([length for length in self.view.shape[:self.view.ndim]]) # <<<<<<<<<<<<<<
+ *
+ * @property
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyList_New(0); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 564, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_3 = (__pyx_v_self->view.shape + __pyx_v_self->view.ndim);
+ for (__pyx_t_4 = __pyx_v_self->view.shape; __pyx_t_4 < __pyx_t_3; __pyx_t_4++) {
+ __pyx_t_2 = __pyx_t_4;
+ __pyx_v_length = (__pyx_t_2[0]);
+ __pyx_t_5 = PyInt_FromSsize_t(__pyx_v_length); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 564, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ if (unlikely(__Pyx_ListComp_Append(__pyx_t_1, (PyObject*)__pyx_t_5))) __PYX_ERR(1, 564, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
+ }
+ __pyx_t_5 = PyList_AsTuple(((PyObject*)__pyx_t_1)); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 564, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_r = __pyx_t_5;
+ __pyx_t_5 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":563
+ *
+ * @property
+ * def shape(self): # <<<<<<<<<<<<<<
+ * return tuple([length for length in self.view.shape[:self.view.ndim]])
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.shape.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":567
+ *
+ * @property
+ * def strides(self): # <<<<<<<<<<<<<<
+ * if self.view.strides == NULL:
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_7strides_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_7strides_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_10memoryview_7strides___get__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_7strides___get__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ Py_ssize_t __pyx_v_stride;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ Py_ssize_t *__pyx_t_3;
+ Py_ssize_t *__pyx_t_4;
+ Py_ssize_t *__pyx_t_5;
+ PyObject *__pyx_t_6 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":568
+ * @property
+ * def strides(self):
+ * if self.view.strides == NULL: # <<<<<<<<<<<<<<
+ *
+ * raise ValueError("Buffer view does not expose strides")
+ */
+ __pyx_t_1 = ((__pyx_v_self->view.strides == NULL) != 0);
+ if (unlikely(__pyx_t_1)) {
+
+ /* "View.MemoryView":570
+ * if self.view.strides == NULL:
+ *
+ * raise ValueError("Buffer view does not expose strides") # <<<<<<<<<<<<<<
+ *
+ * return tuple([stride for stride in self.view.strides[:self.view.ndim]])
+ */
+ __pyx_t_2 = __Pyx_PyObject_Call(__pyx_builtin_ValueError, __pyx_tuple__15, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 570, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_Raise(__pyx_t_2, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __PYX_ERR(1, 570, __pyx_L1_error)
+
+ /* "View.MemoryView":568
+ * @property
+ * def strides(self):
+ * if self.view.strides == NULL: # <<<<<<<<<<<<<<
+ *
+ * raise ValueError("Buffer view does not expose strides")
+ */
+ }
+
+ /* "View.MemoryView":572
+ * raise ValueError("Buffer view does not expose strides")
+ *
+ * return tuple([stride for stride in self.view.strides[:self.view.ndim]]) # <<<<<<<<<<<<<<
+ *
+ * @property
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_2 = PyList_New(0); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 572, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_4 = (__pyx_v_self->view.strides + __pyx_v_self->view.ndim);
+ for (__pyx_t_5 = __pyx_v_self->view.strides; __pyx_t_5 < __pyx_t_4; __pyx_t_5++) {
+ __pyx_t_3 = __pyx_t_5;
+ __pyx_v_stride = (__pyx_t_3[0]);
+ __pyx_t_6 = PyInt_FromSsize_t(__pyx_v_stride); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 572, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ if (unlikely(__Pyx_ListComp_Append(__pyx_t_2, (PyObject*)__pyx_t_6))) __PYX_ERR(1, 572, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
+ }
+ __pyx_t_6 = PyList_AsTuple(((PyObject*)__pyx_t_2)); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 572, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __pyx_r = __pyx_t_6;
+ __pyx_t_6 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":567
+ *
+ * @property
+ * def strides(self): # <<<<<<<<<<<<<<
+ * if self.view.strides == NULL:
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_6);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.strides.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":575
+ *
+ * @property
+ * def suboffsets(self): # <<<<<<<<<<<<<<
+ * if self.view.suboffsets == NULL:
+ * return (-1,) * self.view.ndim
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_10suboffsets_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_10suboffsets_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_10memoryview_10suboffsets___get__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_10suboffsets___get__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ Py_ssize_t __pyx_v_suboffset;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ Py_ssize_t *__pyx_t_4;
+ Py_ssize_t *__pyx_t_5;
+ Py_ssize_t *__pyx_t_6;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":576
+ * @property
+ * def suboffsets(self):
+ * if self.view.suboffsets == NULL: # <<<<<<<<<<<<<<
+ * return (-1,) * self.view.ndim
+ *
+ */
+ __pyx_t_1 = ((__pyx_v_self->view.suboffsets == NULL) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":577
+ * def suboffsets(self):
+ * if self.view.suboffsets == NULL:
+ * return (-1,) * self.view.ndim # <<<<<<<<<<<<<<
+ *
+ * return tuple([suboffset for suboffset in self.view.suboffsets[:self.view.ndim]])
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_2 = __Pyx_PyInt_From_int(__pyx_v_self->view.ndim); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 577, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_3 = PyNumber_Multiply(__pyx_tuple__16, __pyx_t_2); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 577, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __pyx_r = __pyx_t_3;
+ __pyx_t_3 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":576
+ * @property
+ * def suboffsets(self):
+ * if self.view.suboffsets == NULL: # <<<<<<<<<<<<<<
+ * return (-1,) * self.view.ndim
+ *
+ */
+ }
+
+ /* "View.MemoryView":579
+ * return (-1,) * self.view.ndim
+ *
+ * return tuple([suboffset for suboffset in self.view.suboffsets[:self.view.ndim]]) # <<<<<<<<<<<<<<
+ *
+ * @property
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_3 = PyList_New(0); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 579, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_t_5 = (__pyx_v_self->view.suboffsets + __pyx_v_self->view.ndim);
+ for (__pyx_t_6 = __pyx_v_self->view.suboffsets; __pyx_t_6 < __pyx_t_5; __pyx_t_6++) {
+ __pyx_t_4 = __pyx_t_6;
+ __pyx_v_suboffset = (__pyx_t_4[0]);
+ __pyx_t_2 = PyInt_FromSsize_t(__pyx_v_suboffset); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 579, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ if (unlikely(__Pyx_ListComp_Append(__pyx_t_3, (PyObject*)__pyx_t_2))) __PYX_ERR(1, 579, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ }
+ __pyx_t_2 = PyList_AsTuple(((PyObject*)__pyx_t_3)); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 579, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":575
+ *
+ * @property
+ * def suboffsets(self): # <<<<<<<<<<<<<<
+ * if self.view.suboffsets == NULL:
+ * return (-1,) * self.view.ndim
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.suboffsets.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":582
+ *
+ * @property
+ * def ndim(self): # <<<<<<<<<<<<<<
+ * return self.view.ndim
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_4ndim_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_4ndim_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_10memoryview_4ndim___get__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_4ndim___get__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":583
+ * @property
+ * def ndim(self):
+ * return self.view.ndim # <<<<<<<<<<<<<<
+ *
+ * @property
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->view.ndim); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 583, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":582
+ *
+ * @property
+ * def ndim(self): # <<<<<<<<<<<<<<
+ * return self.view.ndim
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.ndim.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":586
+ *
+ * @property
+ * def itemsize(self): # <<<<<<<<<<<<<<
+ * return self.view.itemsize
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_8itemsize_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_8itemsize_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_10memoryview_8itemsize___get__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_8itemsize___get__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":587
+ * @property
+ * def itemsize(self):
+ * return self.view.itemsize # <<<<<<<<<<<<<<
+ *
+ * @property
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = PyInt_FromSsize_t(__pyx_v_self->view.itemsize); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 587, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":586
+ *
+ * @property
+ * def itemsize(self): # <<<<<<<<<<<<<<
+ * return self.view.itemsize
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.itemsize.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":590
+ *
+ * @property
+ * def nbytes(self): # <<<<<<<<<<<<<<
+ * return self.size * self.view.itemsize
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_6nbytes_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_6nbytes_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_10memoryview_6nbytes___get__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_6nbytes___get__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":591
+ * @property
+ * def nbytes(self):
+ * return self.size * self.view.itemsize # <<<<<<<<<<<<<<
+ *
+ * @property
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_self), __pyx_n_s_size); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 591, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_2 = PyInt_FromSsize_t(__pyx_v_self->view.itemsize); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 591, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_3 = PyNumber_Multiply(__pyx_t_1, __pyx_t_2); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 591, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __pyx_r = __pyx_t_3;
+ __pyx_t_3 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":590
+ *
+ * @property
+ * def nbytes(self): # <<<<<<<<<<<<<<
+ * return self.size * self.view.itemsize
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.nbytes.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":594
+ *
+ * @property
+ * def size(self): # <<<<<<<<<<<<<<
+ * if self._size is None:
+ * result = 1
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_4size_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_10memoryview_4size_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_10memoryview_4size___get__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_10memoryview_4size___get__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ PyObject *__pyx_v_result = NULL;
+ PyObject *__pyx_v_length = NULL;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ Py_ssize_t *__pyx_t_3;
+ Py_ssize_t *__pyx_t_4;
+ Py_ssize_t *__pyx_t_5;
+ PyObject *__pyx_t_6 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":595
+ * @property
+ * def size(self):
+ * if self._size is None: # <<<<<<<<<<<<<<
+ * result = 1
+ *
+ */
+ __pyx_t_1 = (__pyx_v_self->_size == Py_None);
+ __pyx_t_2 = (__pyx_t_1 != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":596
+ * def size(self):
+ * if self._size is None:
+ * result = 1 # <<<<<<<<<<<<<<
+ *
+ * for length in self.view.shape[:self.view.ndim]:
+ */
+ __Pyx_INCREF(__pyx_int_1);
+ __pyx_v_result = __pyx_int_1;
+
+ /* "View.MemoryView":598
+ * result = 1
+ *
+ * for length in self.view.shape[:self.view.ndim]: # <<<<<<<<<<<<<<
+ * result *= length
+ *
+ */
+ __pyx_t_4 = (__pyx_v_self->view.shape + __pyx_v_self->view.ndim);
+ for (__pyx_t_5 = __pyx_v_self->view.shape; __pyx_t_5 < __pyx_t_4; __pyx_t_5++) {
+ __pyx_t_3 = __pyx_t_5;
+ __pyx_t_6 = PyInt_FromSsize_t((__pyx_t_3[0])); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 598, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ __Pyx_XDECREF_SET(__pyx_v_length, __pyx_t_6);
+ __pyx_t_6 = 0;
+
+ /* "View.MemoryView":599
+ *
+ * for length in self.view.shape[:self.view.ndim]:
+ * result *= length # <<<<<<<<<<<<<<
+ *
+ * self._size = result
+ */
+ __pyx_t_6 = PyNumber_InPlaceMultiply(__pyx_v_result, __pyx_v_length); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 599, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_6);
+ __Pyx_DECREF_SET(__pyx_v_result, __pyx_t_6);
+ __pyx_t_6 = 0;
+ }
+
+ /* "View.MemoryView":601
+ * result *= length
+ *
+ * self._size = result # <<<<<<<<<<<<<<
+ *
+ * return self._size
+ */
+ __Pyx_INCREF(__pyx_v_result);
+ __Pyx_GIVEREF(__pyx_v_result);
+ __Pyx_GOTREF(__pyx_v_self->_size);
+ __Pyx_DECREF(__pyx_v_self->_size);
+ __pyx_v_self->_size = __pyx_v_result;
+
+ /* "View.MemoryView":595
+ * @property
+ * def size(self):
+ * if self._size is None: # <<<<<<<<<<<<<<
+ * result = 1
+ *
+ */
+ }
+
+ /* "View.MemoryView":603
+ * self._size = result
+ *
+ * return self._size # <<<<<<<<<<<<<<
+ *
+ * def __len__(self):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(__pyx_v_self->_size);
+ __pyx_r = __pyx_v_self->_size;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":594
+ *
+ * @property
+ * def size(self): # <<<<<<<<<<<<<<
+ * if self._size is None:
+ * result = 1
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_6);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.size.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_result);
+ __Pyx_XDECREF(__pyx_v_length);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":605
+ * return self._size
+ *
+ * def __len__(self): # <<<<<<<<<<<<<<
+ * if self.view.ndim >= 1:
+ * return self.view.shape[0]
+ */
+
+/* Python wrapper */
+static Py_ssize_t __pyx_memoryview___len__(PyObject *__pyx_v_self); /*proto*/
+static Py_ssize_t __pyx_memoryview___len__(PyObject *__pyx_v_self) {
+ Py_ssize_t __pyx_r;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__len__ (wrapper)", 0);
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_10__len__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static Py_ssize_t __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_10__len__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ Py_ssize_t __pyx_r;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ __Pyx_RefNannySetupContext("__len__", 0);
+
+ /* "View.MemoryView":606
+ *
+ * def __len__(self):
+ * if self.view.ndim >= 1: # <<<<<<<<<<<<<<
+ * return self.view.shape[0]
+ *
+ */
+ __pyx_t_1 = ((__pyx_v_self->view.ndim >= 1) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":607
+ * def __len__(self):
+ * if self.view.ndim >= 1:
+ * return self.view.shape[0] # <<<<<<<<<<<<<<
+ *
+ * return 0
+ */
+ __pyx_r = (__pyx_v_self->view.shape[0]);
+ goto __pyx_L0;
+
+ /* "View.MemoryView":606
+ *
+ * def __len__(self):
+ * if self.view.ndim >= 1: # <<<<<<<<<<<<<<
+ * return self.view.shape[0]
+ *
+ */
+ }
+
+ /* "View.MemoryView":609
+ * return self.view.shape[0]
+ *
+ * return 0 # <<<<<<<<<<<<<<
+ *
+ * def __repr__(self):
+ */
+ __pyx_r = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":605
+ * return self._size
+ *
+ * def __len__(self): # <<<<<<<<<<<<<<
+ * if self.view.ndim >= 1:
+ * return self.view.shape[0]
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":611
+ * return 0
+ *
+ * def __repr__(self): # <<<<<<<<<<<<<<
+ * return "" % (self.base.__class__.__name__,
+ * id(self))
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_memoryview___repr__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_memoryview___repr__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__repr__ (wrapper)", 0);
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_12__repr__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_12__repr__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__repr__", 0);
+
+ /* "View.MemoryView":612
+ *
+ * def __repr__(self):
+ * return "" % (self.base.__class__.__name__, # <<<<<<<<<<<<<<
+ * id(self))
+ *
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_self), __pyx_n_s_base); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 612, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_class); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 612, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_name_2); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 612, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+
+ /* "View.MemoryView":613
+ * def __repr__(self):
+ * return "" % (self.base.__class__.__name__,
+ * id(self)) # <<<<<<<<<<<<<<
+ *
+ * def __str__(self):
+ */
+ __pyx_t_2 = __Pyx_PyObject_CallOneArg(__pyx_builtin_id, ((PyObject *)__pyx_v_self)); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 613, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+
+ /* "View.MemoryView":612
+ *
+ * def __repr__(self):
+ * return "" % (self.base.__class__.__name__, # <<<<<<<<<<<<<<
+ * id(self))
+ *
+ */
+ __pyx_t_3 = PyTuple_New(2); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 612, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_GIVEREF(__pyx_t_1);
+ PyTuple_SET_ITEM(__pyx_t_3, 0, __pyx_t_1);
+ __Pyx_GIVEREF(__pyx_t_2);
+ PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_t_2);
+ __pyx_t_1 = 0;
+ __pyx_t_2 = 0;
+ __pyx_t_2 = __Pyx_PyString_Format(__pyx_kp_s_MemoryView_of_r_at_0x_x, __pyx_t_3); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 612, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":611
+ * return 0
+ *
+ * def __repr__(self): # <<<<<<<<<<<<<<
+ * return "" % (self.base.__class__.__name__,
+ * id(self))
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.__repr__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":615
+ * id(self))
+ *
+ * def __str__(self): # <<<<<<<<<<<<<<
+ * return "" % (self.base.__class__.__name__,)
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_memoryview___str__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_memoryview___str__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__str__ (wrapper)", 0);
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_14__str__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_14__str__(struct __pyx_memoryview_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__str__", 0);
+
+ /* "View.MemoryView":616
+ *
+ * def __str__(self):
+ * return "" % (self.base.__class__.__name__,) # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_self), __pyx_n_s_base); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 616, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_class); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 616, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_name_2); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 616, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __pyx_t_2 = PyTuple_New(1); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 616, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_GIVEREF(__pyx_t_1);
+ PyTuple_SET_ITEM(__pyx_t_2, 0, __pyx_t_1);
+ __pyx_t_1 = 0;
+ __pyx_t_1 = __Pyx_PyString_Format(__pyx_kp_s_MemoryView_of_r_object, __pyx_t_2); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 616, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":615
+ * id(self))
+ *
+ * def __str__(self): # <<<<<<<<<<<<<<
+ * return "" % (self.base.__class__.__name__,)
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.__str__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":619
+ *
+ *
+ * def is_c_contig(self): # <<<<<<<<<<<<<<
+ * cdef __Pyx_memviewslice *mslice
+ * cdef __Pyx_memviewslice tmp
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_memoryview_is_c_contig(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/
+static PyObject *__pyx_memoryview_is_c_contig(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("is_c_contig (wrapper)", 0);
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_16is_c_contig(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_16is_c_contig(struct __pyx_memoryview_obj *__pyx_v_self) {
+ __Pyx_memviewslice *__pyx_v_mslice;
+ __Pyx_memviewslice __pyx_v_tmp;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ __Pyx_memviewslice *__pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("is_c_contig", 0);
+
+ /* "View.MemoryView":622
+ * cdef __Pyx_memviewslice *mslice
+ * cdef __Pyx_memviewslice tmp
+ * mslice = get_slice_from_memview(self, &tmp) # <<<<<<<<<<<<<<
+ * return slice_is_contig(mslice[0], 'C', self.view.ndim)
+ *
+ */
+ __pyx_t_1 = __pyx_memoryview_get_slice_from_memoryview(__pyx_v_self, (&__pyx_v_tmp)); if (unlikely(__pyx_t_1 == ((__Pyx_memviewslice *)NULL))) __PYX_ERR(1, 622, __pyx_L1_error)
+ __pyx_v_mslice = __pyx_t_1;
+
+ /* "View.MemoryView":623
+ * cdef __Pyx_memviewslice tmp
+ * mslice = get_slice_from_memview(self, &tmp)
+ * return slice_is_contig(mslice[0], 'C', self.view.ndim) # <<<<<<<<<<<<<<
+ *
+ * def is_f_contig(self):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_2 = __Pyx_PyBool_FromLong(__pyx_memviewslice_is_contig((__pyx_v_mslice[0]), 'C', __pyx_v_self->view.ndim)); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 623, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":619
+ *
+ *
+ * def is_c_contig(self): # <<<<<<<<<<<<<<
+ * cdef __Pyx_memviewslice *mslice
+ * cdef __Pyx_memviewslice tmp
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.is_c_contig", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":625
+ * return slice_is_contig(mslice[0], 'C', self.view.ndim)
+ *
+ * def is_f_contig(self): # <<<<<<<<<<<<<<
+ * cdef __Pyx_memviewslice *mslice
+ * cdef __Pyx_memviewslice tmp
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_memoryview_is_f_contig(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/
+static PyObject *__pyx_memoryview_is_f_contig(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("is_f_contig (wrapper)", 0);
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_18is_f_contig(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_18is_f_contig(struct __pyx_memoryview_obj *__pyx_v_self) {
+ __Pyx_memviewslice *__pyx_v_mslice;
+ __Pyx_memviewslice __pyx_v_tmp;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ __Pyx_memviewslice *__pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("is_f_contig", 0);
+
+ /* "View.MemoryView":628
+ * cdef __Pyx_memviewslice *mslice
+ * cdef __Pyx_memviewslice tmp
+ * mslice = get_slice_from_memview(self, &tmp) # <<<<<<<<<<<<<<
+ * return slice_is_contig(mslice[0], 'F', self.view.ndim)
+ *
+ */
+ __pyx_t_1 = __pyx_memoryview_get_slice_from_memoryview(__pyx_v_self, (&__pyx_v_tmp)); if (unlikely(__pyx_t_1 == ((__Pyx_memviewslice *)NULL))) __PYX_ERR(1, 628, __pyx_L1_error)
+ __pyx_v_mslice = __pyx_t_1;
+
+ /* "View.MemoryView":629
+ * cdef __Pyx_memviewslice tmp
+ * mslice = get_slice_from_memview(self, &tmp)
+ * return slice_is_contig(mslice[0], 'F', self.view.ndim) # <<<<<<<<<<<<<<
+ *
+ * def copy(self):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_2 = __Pyx_PyBool_FromLong(__pyx_memviewslice_is_contig((__pyx_v_mslice[0]), 'F', __pyx_v_self->view.ndim)); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 629, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":625
+ * return slice_is_contig(mslice[0], 'C', self.view.ndim)
+ *
+ * def is_f_contig(self): # <<<<<<<<<<<<<<
+ * cdef __Pyx_memviewslice *mslice
+ * cdef __Pyx_memviewslice tmp
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.is_f_contig", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":631
+ * return slice_is_contig(mslice[0], 'F', self.view.ndim)
+ *
+ * def copy(self): # <<<<<<<<<<<<<<
+ * cdef __Pyx_memviewslice mslice
+ * cdef int flags = self.flags & ~PyBUF_F_CONTIGUOUS
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_memoryview_copy(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/
+static PyObject *__pyx_memoryview_copy(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("copy (wrapper)", 0);
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_20copy(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_20copy(struct __pyx_memoryview_obj *__pyx_v_self) {
+ __Pyx_memviewslice __pyx_v_mslice;
+ int __pyx_v_flags;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ __Pyx_memviewslice __pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("copy", 0);
+
+ /* "View.MemoryView":633
+ * def copy(self):
+ * cdef __Pyx_memviewslice mslice
+ * cdef int flags = self.flags & ~PyBUF_F_CONTIGUOUS # <<<<<<<<<<<<<<
+ *
+ * slice_copy(self, &mslice)
+ */
+ __pyx_v_flags = (__pyx_v_self->flags & (~PyBUF_F_CONTIGUOUS));
+
+ /* "View.MemoryView":635
+ * cdef int flags = self.flags & ~PyBUF_F_CONTIGUOUS
+ *
+ * slice_copy(self, &mslice) # <<<<<<<<<<<<<<
+ * mslice = slice_copy_contig(&mslice, "c", self.view.ndim,
+ * self.view.itemsize,
+ */
+ __pyx_memoryview_slice_copy(__pyx_v_self, (&__pyx_v_mslice));
+
+ /* "View.MemoryView":636
+ *
+ * slice_copy(self, &mslice)
+ * mslice = slice_copy_contig(&mslice, "c", self.view.ndim, # <<<<<<<<<<<<<<
+ * self.view.itemsize,
+ * flags|PyBUF_C_CONTIGUOUS,
+ */
+ __pyx_t_1 = __pyx_memoryview_copy_new_contig((&__pyx_v_mslice), ((char *)"c"), __pyx_v_self->view.ndim, __pyx_v_self->view.itemsize, (__pyx_v_flags | PyBUF_C_CONTIGUOUS), __pyx_v_self->dtype_is_object); if (unlikely(PyErr_Occurred())) __PYX_ERR(1, 636, __pyx_L1_error)
+ __pyx_v_mslice = __pyx_t_1;
+
+ /* "View.MemoryView":641
+ * self.dtype_is_object)
+ *
+ * return memoryview_copy_from_slice(self, &mslice) # <<<<<<<<<<<<<<
+ *
+ * def copy_fortran(self):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_2 = __pyx_memoryview_copy_object_from_slice(__pyx_v_self, (&__pyx_v_mslice)); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 641, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":631
+ * return slice_is_contig(mslice[0], 'F', self.view.ndim)
+ *
+ * def copy(self): # <<<<<<<<<<<<<<
+ * cdef __Pyx_memviewslice mslice
+ * cdef int flags = self.flags & ~PyBUF_F_CONTIGUOUS
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.copy", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":643
+ * return memoryview_copy_from_slice(self, &mslice)
+ *
+ * def copy_fortran(self): # <<<<<<<<<<<<<<
+ * cdef __Pyx_memviewslice src, dst
+ * cdef int flags = self.flags & ~PyBUF_C_CONTIGUOUS
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_memoryview_copy_fortran(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/
+static PyObject *__pyx_memoryview_copy_fortran(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("copy_fortran (wrapper)", 0);
+ __pyx_r = __pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_22copy_fortran(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_memoryview___pyx_pf_15View_dot_MemoryView_10memoryview_22copy_fortran(struct __pyx_memoryview_obj *__pyx_v_self) {
+ __Pyx_memviewslice __pyx_v_src;
+ __Pyx_memviewslice __pyx_v_dst;
+ int __pyx_v_flags;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ __Pyx_memviewslice __pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("copy_fortran", 0);
+
+ /* "View.MemoryView":645
+ * def copy_fortran(self):
+ * cdef __Pyx_memviewslice src, dst
+ * cdef int flags = self.flags & ~PyBUF_C_CONTIGUOUS # <<<<<<<<<<<<<<
+ *
+ * slice_copy(self, &src)
+ */
+ __pyx_v_flags = (__pyx_v_self->flags & (~PyBUF_C_CONTIGUOUS));
+
+ /* "View.MemoryView":647
+ * cdef int flags = self.flags & ~PyBUF_C_CONTIGUOUS
+ *
+ * slice_copy(self, &src) # <<<<<<<<<<<<<<
+ * dst = slice_copy_contig(&src, "fortran", self.view.ndim,
+ * self.view.itemsize,
+ */
+ __pyx_memoryview_slice_copy(__pyx_v_self, (&__pyx_v_src));
+
+ /* "View.MemoryView":648
+ *
+ * slice_copy(self, &src)
+ * dst = slice_copy_contig(&src, "fortran", self.view.ndim, # <<<<<<<<<<<<<<
+ * self.view.itemsize,
+ * flags|PyBUF_F_CONTIGUOUS,
+ */
+ __pyx_t_1 = __pyx_memoryview_copy_new_contig((&__pyx_v_src), ((char *)"fortran"), __pyx_v_self->view.ndim, __pyx_v_self->view.itemsize, (__pyx_v_flags | PyBUF_F_CONTIGUOUS), __pyx_v_self->dtype_is_object); if (unlikely(PyErr_Occurred())) __PYX_ERR(1, 648, __pyx_L1_error)
+ __pyx_v_dst = __pyx_t_1;
+
+ /* "View.MemoryView":653
+ * self.dtype_is_object)
+ *
+ * return memoryview_copy_from_slice(self, &dst) # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_2 = __pyx_memoryview_copy_object_from_slice(__pyx_v_self, (&__pyx_v_dst)); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 653, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":643
+ * return memoryview_copy_from_slice(self, &mslice)
+ *
+ * def copy_fortran(self): # <<<<<<<<<<<<<<
+ * cdef __Pyx_memviewslice src, dst
+ * cdef int flags = self.flags & ~PyBUF_C_CONTIGUOUS
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.copy_fortran", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "(tree fragment)":1
+ * def __reduce_cython__(self): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw___pyx_memoryview_1__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/
+static PyObject *__pyx_pw___pyx_memoryview_1__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0);
+ __pyx_r = __pyx_pf___pyx_memoryview___reduce_cython__(((struct __pyx_memoryview_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf___pyx_memoryview___reduce_cython__(CYTHON_UNUSED struct __pyx_memoryview_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__reduce_cython__", 0);
+
+ /* "(tree fragment)":2
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<<
+ * def __setstate_cython__(self, __pyx_state):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__17, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 2, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_Raise(__pyx_t_1, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __PYX_ERR(1, 2, __pyx_L1_error)
+
+ /* "(tree fragment)":1
+ * def __reduce_cython__(self): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "(tree fragment)":3
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw___pyx_memoryview_3__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/
+static PyObject *__pyx_pw___pyx_memoryview_3__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0);
+ __pyx_r = __pyx_pf___pyx_memoryview_2__setstate_cython__(((struct __pyx_memoryview_obj *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf___pyx_memoryview_2__setstate_cython__(CYTHON_UNUSED struct __pyx_memoryview_obj *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__setstate_cython__", 0);
+
+ /* "(tree fragment)":4
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<<
+ */
+ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__18, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_Raise(__pyx_t_1, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __PYX_ERR(1, 4, __pyx_L1_error)
+
+ /* "(tree fragment)":3
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.memoryview.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":657
+ *
+ * @cname('__pyx_memoryview_new')
+ * cdef memoryview_cwrapper(object o, int flags, bint dtype_is_object, __Pyx_TypeInfo *typeinfo): # <<<<<<<<<<<<<<
+ * cdef memoryview result = memoryview(o, flags, dtype_is_object)
+ * result.typeinfo = typeinfo
+ */
+
+static PyObject *__pyx_memoryview_new(PyObject *__pyx_v_o, int __pyx_v_flags, int __pyx_v_dtype_is_object, __Pyx_TypeInfo *__pyx_v_typeinfo) {
+ struct __pyx_memoryview_obj *__pyx_v_result = 0;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("memoryview_cwrapper", 0);
+
+ /* "View.MemoryView":658
+ * @cname('__pyx_memoryview_new')
+ * cdef memoryview_cwrapper(object o, int flags, bint dtype_is_object, __Pyx_TypeInfo *typeinfo):
+ * cdef memoryview result = memoryview(o, flags, dtype_is_object) # <<<<<<<<<<<<<<
+ * result.typeinfo = typeinfo
+ * return result
+ */
+ __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_flags); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 658, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_t_2 = __Pyx_PyBool_FromLong(__pyx_v_dtype_is_object); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 658, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_3 = PyTuple_New(3); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 658, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_INCREF(__pyx_v_o);
+ __Pyx_GIVEREF(__pyx_v_o);
+ PyTuple_SET_ITEM(__pyx_t_3, 0, __pyx_v_o);
+ __Pyx_GIVEREF(__pyx_t_1);
+ PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_t_1);
+ __Pyx_GIVEREF(__pyx_t_2);
+ PyTuple_SET_ITEM(__pyx_t_3, 2, __pyx_t_2);
+ __pyx_t_1 = 0;
+ __pyx_t_2 = 0;
+ __pyx_t_2 = __Pyx_PyObject_Call(((PyObject *)__pyx_memoryview_type), __pyx_t_3, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 658, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __pyx_v_result = ((struct __pyx_memoryview_obj *)__pyx_t_2);
+ __pyx_t_2 = 0;
+
+ /* "View.MemoryView":659
+ * cdef memoryview_cwrapper(object o, int flags, bint dtype_is_object, __Pyx_TypeInfo *typeinfo):
+ * cdef memoryview result = memoryview(o, flags, dtype_is_object)
+ * result.typeinfo = typeinfo # <<<<<<<<<<<<<<
+ * return result
+ *
+ */
+ __pyx_v_result->typeinfo = __pyx_v_typeinfo;
+
+ /* "View.MemoryView":660
+ * cdef memoryview result = memoryview(o, flags, dtype_is_object)
+ * result.typeinfo = typeinfo
+ * return result # <<<<<<<<<<<<<<
+ *
+ * @cname('__pyx_memoryview_check')
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(((PyObject *)__pyx_v_result));
+ __pyx_r = ((PyObject *)__pyx_v_result);
+ goto __pyx_L0;
+
+ /* "View.MemoryView":657
+ *
+ * @cname('__pyx_memoryview_new')
+ * cdef memoryview_cwrapper(object o, int flags, bint dtype_is_object, __Pyx_TypeInfo *typeinfo): # <<<<<<<<<<<<<<
+ * cdef memoryview result = memoryview(o, flags, dtype_is_object)
+ * result.typeinfo = typeinfo
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.memoryview_cwrapper", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XDECREF((PyObject *)__pyx_v_result);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":663
+ *
+ * @cname('__pyx_memoryview_check')
+ * cdef inline bint memoryview_check(object o): # <<<<<<<<<<<<<<
+ * return isinstance(o, memoryview)
+ *
+ */
+
+static CYTHON_INLINE int __pyx_memoryview_check(PyObject *__pyx_v_o) {
+ int __pyx_r;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ __Pyx_RefNannySetupContext("memoryview_check", 0);
+
+ /* "View.MemoryView":664
+ * @cname('__pyx_memoryview_check')
+ * cdef inline bint memoryview_check(object o):
+ * return isinstance(o, memoryview) # <<<<<<<<<<<<<<
+ *
+ * cdef tuple _unellipsify(object index, int ndim):
+ */
+ __pyx_t_1 = __Pyx_TypeCheck(__pyx_v_o, __pyx_memoryview_type);
+ __pyx_r = __pyx_t_1;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":663
+ *
+ * @cname('__pyx_memoryview_check')
+ * cdef inline bint memoryview_check(object o): # <<<<<<<<<<<<<<
+ * return isinstance(o, memoryview)
+ *
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":666
+ * return isinstance(o, memoryview)
+ *
+ * cdef tuple _unellipsify(object index, int ndim): # <<<<<<<<<<<<<<
+ * """
+ * Replace all ellipses with full slices and fill incomplete indices with
+ */
+
+static PyObject *_unellipsify(PyObject *__pyx_v_index, int __pyx_v_ndim) {
+ PyObject *__pyx_v_tup = NULL;
+ PyObject *__pyx_v_result = NULL;
+ int __pyx_v_have_slices;
+ int __pyx_v_seen_ellipsis;
+ CYTHON_UNUSED PyObject *__pyx_v_idx = NULL;
+ PyObject *__pyx_v_item = NULL;
+ Py_ssize_t __pyx_v_nslices;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ PyObject *__pyx_t_4 = NULL;
+ Py_ssize_t __pyx_t_5;
+ PyObject *(*__pyx_t_6)(PyObject *);
+ PyObject *__pyx_t_7 = NULL;
+ Py_ssize_t __pyx_t_8;
+ int __pyx_t_9;
+ int __pyx_t_10;
+ PyObject *__pyx_t_11 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("_unellipsify", 0);
+
+ /* "View.MemoryView":671
+ * full slices.
+ * """
+ * if not isinstance(index, tuple): # <<<<<<<<<<<<<<
+ * tup = (index,)
+ * else:
+ */
+ __pyx_t_1 = PyTuple_Check(__pyx_v_index);
+ __pyx_t_2 = ((!(__pyx_t_1 != 0)) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":672
+ * """
+ * if not isinstance(index, tuple):
+ * tup = (index,) # <<<<<<<<<<<<<<
+ * else:
+ * tup = index
+ */
+ __pyx_t_3 = PyTuple_New(1); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 672, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_INCREF(__pyx_v_index);
+ __Pyx_GIVEREF(__pyx_v_index);
+ PyTuple_SET_ITEM(__pyx_t_3, 0, __pyx_v_index);
+ __pyx_v_tup = __pyx_t_3;
+ __pyx_t_3 = 0;
+
+ /* "View.MemoryView":671
+ * full slices.
+ * """
+ * if not isinstance(index, tuple): # <<<<<<<<<<<<<<
+ * tup = (index,)
+ * else:
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":674
+ * tup = (index,)
+ * else:
+ * tup = index # <<<<<<<<<<<<<<
+ *
+ * result = []
+ */
+ /*else*/ {
+ __Pyx_INCREF(__pyx_v_index);
+ __pyx_v_tup = __pyx_v_index;
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":676
+ * tup = index
+ *
+ * result = [] # <<<<<<<<<<<<<<
+ * have_slices = False
+ * seen_ellipsis = False
+ */
+ __pyx_t_3 = PyList_New(0); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 676, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_v_result = ((PyObject*)__pyx_t_3);
+ __pyx_t_3 = 0;
+
+ /* "View.MemoryView":677
+ *
+ * result = []
+ * have_slices = False # <<<<<<<<<<<<<<
+ * seen_ellipsis = False
+ * for idx, item in enumerate(tup):
+ */
+ __pyx_v_have_slices = 0;
+
+ /* "View.MemoryView":678
+ * result = []
+ * have_slices = False
+ * seen_ellipsis = False # <<<<<<<<<<<<<<
+ * for idx, item in enumerate(tup):
+ * if item is Ellipsis:
+ */
+ __pyx_v_seen_ellipsis = 0;
+
+ /* "View.MemoryView":679
+ * have_slices = False
+ * seen_ellipsis = False
+ * for idx, item in enumerate(tup): # <<<<<<<<<<<<<<
+ * if item is Ellipsis:
+ * if not seen_ellipsis:
+ */
+ __Pyx_INCREF(__pyx_int_0);
+ __pyx_t_3 = __pyx_int_0;
+ if (likely(PyList_CheckExact(__pyx_v_tup)) || PyTuple_CheckExact(__pyx_v_tup)) {
+ __pyx_t_4 = __pyx_v_tup; __Pyx_INCREF(__pyx_t_4); __pyx_t_5 = 0;
+ __pyx_t_6 = NULL;
+ } else {
+ __pyx_t_5 = -1; __pyx_t_4 = PyObject_GetIter(__pyx_v_tup); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 679, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_6 = Py_TYPE(__pyx_t_4)->tp_iternext; if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 679, __pyx_L1_error)
+ }
+ for (;;) {
+ if (likely(!__pyx_t_6)) {
+ if (likely(PyList_CheckExact(__pyx_t_4))) {
+ if (__pyx_t_5 >= PyList_GET_SIZE(__pyx_t_4)) break;
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_7 = PyList_GET_ITEM(__pyx_t_4, __pyx_t_5); __Pyx_INCREF(__pyx_t_7); __pyx_t_5++; if (unlikely(0 < 0)) __PYX_ERR(1, 679, __pyx_L1_error)
+ #else
+ __pyx_t_7 = PySequence_ITEM(__pyx_t_4, __pyx_t_5); __pyx_t_5++; if (unlikely(!__pyx_t_7)) __PYX_ERR(1, 679, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_7);
+ #endif
+ } else {
+ if (__pyx_t_5 >= PyTuple_GET_SIZE(__pyx_t_4)) break;
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_7 = PyTuple_GET_ITEM(__pyx_t_4, __pyx_t_5); __Pyx_INCREF(__pyx_t_7); __pyx_t_5++; if (unlikely(0 < 0)) __PYX_ERR(1, 679, __pyx_L1_error)
+ #else
+ __pyx_t_7 = PySequence_ITEM(__pyx_t_4, __pyx_t_5); __pyx_t_5++; if (unlikely(!__pyx_t_7)) __PYX_ERR(1, 679, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_7);
+ #endif
+ }
+ } else {
+ __pyx_t_7 = __pyx_t_6(__pyx_t_4);
+ if (unlikely(!__pyx_t_7)) {
+ PyObject* exc_type = PyErr_Occurred();
+ if (exc_type) {
+ if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear();
+ else __PYX_ERR(1, 679, __pyx_L1_error)
+ }
+ break;
+ }
+ __Pyx_GOTREF(__pyx_t_7);
+ }
+ __Pyx_XDECREF_SET(__pyx_v_item, __pyx_t_7);
+ __pyx_t_7 = 0;
+ __Pyx_INCREF(__pyx_t_3);
+ __Pyx_XDECREF_SET(__pyx_v_idx, __pyx_t_3);
+ __pyx_t_7 = __Pyx_PyInt_AddObjC(__pyx_t_3, __pyx_int_1, 1, 0, 0); if (unlikely(!__pyx_t_7)) __PYX_ERR(1, 679, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_7);
+ __Pyx_DECREF(__pyx_t_3);
+ __pyx_t_3 = __pyx_t_7;
+ __pyx_t_7 = 0;
+
+ /* "View.MemoryView":680
+ * seen_ellipsis = False
+ * for idx, item in enumerate(tup):
+ * if item is Ellipsis: # <<<<<<<<<<<<<<
+ * if not seen_ellipsis:
+ * result.extend([slice(None)] * (ndim - len(tup) + 1))
+ */
+ __pyx_t_2 = (__pyx_v_item == __pyx_builtin_Ellipsis);
+ __pyx_t_1 = (__pyx_t_2 != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":681
+ * for idx, item in enumerate(tup):
+ * if item is Ellipsis:
+ * if not seen_ellipsis: # <<<<<<<<<<<<<<
+ * result.extend([slice(None)] * (ndim - len(tup) + 1))
+ * seen_ellipsis = True
+ */
+ __pyx_t_1 = ((!(__pyx_v_seen_ellipsis != 0)) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":682
+ * if item is Ellipsis:
+ * if not seen_ellipsis:
+ * result.extend([slice(None)] * (ndim - len(tup) + 1)) # <<<<<<<<<<<<<<
+ * seen_ellipsis = True
+ * else:
+ */
+ __pyx_t_8 = PyObject_Length(__pyx_v_tup); if (unlikely(__pyx_t_8 == ((Py_ssize_t)-1))) __PYX_ERR(1, 682, __pyx_L1_error)
+ __pyx_t_7 = PyList_New(1 * ((((__pyx_v_ndim - __pyx_t_8) + 1)<0) ? 0:((__pyx_v_ndim - __pyx_t_8) + 1))); if (unlikely(!__pyx_t_7)) __PYX_ERR(1, 682, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_7);
+ { Py_ssize_t __pyx_temp;
+ for (__pyx_temp=0; __pyx_temp < ((__pyx_v_ndim - __pyx_t_8) + 1); __pyx_temp++) {
+ __Pyx_INCREF(__pyx_slice__19);
+ __Pyx_GIVEREF(__pyx_slice__19);
+ PyList_SET_ITEM(__pyx_t_7, __pyx_temp, __pyx_slice__19);
+ }
+ }
+ __pyx_t_9 = __Pyx_PyList_Extend(__pyx_v_result, __pyx_t_7); if (unlikely(__pyx_t_9 == ((int)-1))) __PYX_ERR(1, 682, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0;
+
+ /* "View.MemoryView":683
+ * if not seen_ellipsis:
+ * result.extend([slice(None)] * (ndim - len(tup) + 1))
+ * seen_ellipsis = True # <<<<<<<<<<<<<<
+ * else:
+ * result.append(slice(None))
+ */
+ __pyx_v_seen_ellipsis = 1;
+
+ /* "View.MemoryView":681
+ * for idx, item in enumerate(tup):
+ * if item is Ellipsis:
+ * if not seen_ellipsis: # <<<<<<<<<<<<<<
+ * result.extend([slice(None)] * (ndim - len(tup) + 1))
+ * seen_ellipsis = True
+ */
+ goto __pyx_L7;
+ }
+
+ /* "View.MemoryView":685
+ * seen_ellipsis = True
+ * else:
+ * result.append(slice(None)) # <<<<<<<<<<<<<<
+ * have_slices = True
+ * else:
+ */
+ /*else*/ {
+ __pyx_t_9 = __Pyx_PyList_Append(__pyx_v_result, __pyx_slice__19); if (unlikely(__pyx_t_9 == ((int)-1))) __PYX_ERR(1, 685, __pyx_L1_error)
+ }
+ __pyx_L7:;
+
+ /* "View.MemoryView":686
+ * else:
+ * result.append(slice(None))
+ * have_slices = True # <<<<<<<<<<<<<<
+ * else:
+ * if not isinstance(item, slice) and not PyIndex_Check(item):
+ */
+ __pyx_v_have_slices = 1;
+
+ /* "View.MemoryView":680
+ * seen_ellipsis = False
+ * for idx, item in enumerate(tup):
+ * if item is Ellipsis: # <<<<<<<<<<<<<<
+ * if not seen_ellipsis:
+ * result.extend([slice(None)] * (ndim - len(tup) + 1))
+ */
+ goto __pyx_L6;
+ }
+
+ /* "View.MemoryView":688
+ * have_slices = True
+ * else:
+ * if not isinstance(item, slice) and not PyIndex_Check(item): # <<<<<<<<<<<<<<
+ * raise TypeError("Cannot index with type '%s'" % type(item))
+ *
+ */
+ /*else*/ {
+ __pyx_t_2 = PySlice_Check(__pyx_v_item);
+ __pyx_t_10 = ((!(__pyx_t_2 != 0)) != 0);
+ if (__pyx_t_10) {
+ } else {
+ __pyx_t_1 = __pyx_t_10;
+ goto __pyx_L9_bool_binop_done;
+ }
+ __pyx_t_10 = ((!(PyIndex_Check(__pyx_v_item) != 0)) != 0);
+ __pyx_t_1 = __pyx_t_10;
+ __pyx_L9_bool_binop_done:;
+ if (unlikely(__pyx_t_1)) {
+
+ /* "View.MemoryView":689
+ * else:
+ * if not isinstance(item, slice) and not PyIndex_Check(item):
+ * raise TypeError("Cannot index with type '%s'" % type(item)) # <<<<<<<<<<<<<<
+ *
+ * have_slices = have_slices or isinstance(item, slice)
+ */
+ __pyx_t_7 = __Pyx_PyString_FormatSafe(__pyx_kp_s_Cannot_index_with_type_s, ((PyObject *)Py_TYPE(__pyx_v_item))); if (unlikely(!__pyx_t_7)) __PYX_ERR(1, 689, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_7);
+ __pyx_t_11 = __Pyx_PyObject_CallOneArg(__pyx_builtin_TypeError, __pyx_t_7); if (unlikely(!__pyx_t_11)) __PYX_ERR(1, 689, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_11);
+ __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0;
+ __Pyx_Raise(__pyx_t_11, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_11); __pyx_t_11 = 0;
+ __PYX_ERR(1, 689, __pyx_L1_error)
+
+ /* "View.MemoryView":688
+ * have_slices = True
+ * else:
+ * if not isinstance(item, slice) and not PyIndex_Check(item): # <<<<<<<<<<<<<<
+ * raise TypeError("Cannot index with type '%s'" % type(item))
+ *
+ */
+ }
+
+ /* "View.MemoryView":691
+ * raise TypeError("Cannot index with type '%s'" % type(item))
+ *
+ * have_slices = have_slices or isinstance(item, slice) # <<<<<<<<<<<<<<
+ * result.append(item)
+ *
+ */
+ __pyx_t_10 = (__pyx_v_have_slices != 0);
+ if (!__pyx_t_10) {
+ } else {
+ __pyx_t_1 = __pyx_t_10;
+ goto __pyx_L11_bool_binop_done;
+ }
+ __pyx_t_10 = PySlice_Check(__pyx_v_item);
+ __pyx_t_2 = (__pyx_t_10 != 0);
+ __pyx_t_1 = __pyx_t_2;
+ __pyx_L11_bool_binop_done:;
+ __pyx_v_have_slices = __pyx_t_1;
+
+ /* "View.MemoryView":692
+ *
+ * have_slices = have_slices or isinstance(item, slice)
+ * result.append(item) # <<<<<<<<<<<<<<
+ *
+ * nslices = ndim - len(result)
+ */
+ __pyx_t_9 = __Pyx_PyList_Append(__pyx_v_result, __pyx_v_item); if (unlikely(__pyx_t_9 == ((int)-1))) __PYX_ERR(1, 692, __pyx_L1_error)
+ }
+ __pyx_L6:;
+
+ /* "View.MemoryView":679
+ * have_slices = False
+ * seen_ellipsis = False
+ * for idx, item in enumerate(tup): # <<<<<<<<<<<<<<
+ * if item is Ellipsis:
+ * if not seen_ellipsis:
+ */
+ }
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+
+ /* "View.MemoryView":694
+ * result.append(item)
+ *
+ * nslices = ndim - len(result) # <<<<<<<<<<<<<<
+ * if nslices:
+ * result.extend([slice(None)] * nslices)
+ */
+ __pyx_t_5 = PyList_GET_SIZE(__pyx_v_result); if (unlikely(__pyx_t_5 == ((Py_ssize_t)-1))) __PYX_ERR(1, 694, __pyx_L1_error)
+ __pyx_v_nslices = (__pyx_v_ndim - __pyx_t_5);
+
+ /* "View.MemoryView":695
+ *
+ * nslices = ndim - len(result)
+ * if nslices: # <<<<<<<<<<<<<<
+ * result.extend([slice(None)] * nslices)
+ *
+ */
+ __pyx_t_1 = (__pyx_v_nslices != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":696
+ * nslices = ndim - len(result)
+ * if nslices:
+ * result.extend([slice(None)] * nslices) # <<<<<<<<<<<<<<
+ *
+ * return have_slices or nslices, tuple(result)
+ */
+ __pyx_t_3 = PyList_New(1 * ((__pyx_v_nslices<0) ? 0:__pyx_v_nslices)); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 696, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ { Py_ssize_t __pyx_temp;
+ for (__pyx_temp=0; __pyx_temp < __pyx_v_nslices; __pyx_temp++) {
+ __Pyx_INCREF(__pyx_slice__19);
+ __Pyx_GIVEREF(__pyx_slice__19);
+ PyList_SET_ITEM(__pyx_t_3, __pyx_temp, __pyx_slice__19);
+ }
+ }
+ __pyx_t_9 = __Pyx_PyList_Extend(__pyx_v_result, __pyx_t_3); if (unlikely(__pyx_t_9 == ((int)-1))) __PYX_ERR(1, 696, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+
+ /* "View.MemoryView":695
+ *
+ * nslices = ndim - len(result)
+ * if nslices: # <<<<<<<<<<<<<<
+ * result.extend([slice(None)] * nslices)
+ *
+ */
+ }
+
+ /* "View.MemoryView":698
+ * result.extend([slice(None)] * nslices)
+ *
+ * return have_slices or nslices, tuple(result) # <<<<<<<<<<<<<<
+ *
+ * cdef assert_direct_dimensions(Py_ssize_t *suboffsets, int ndim):
+ */
+ __Pyx_XDECREF(__pyx_r);
+ if (!__pyx_v_have_slices) {
+ } else {
+ __pyx_t_4 = __Pyx_PyBool_FromLong(__pyx_v_have_slices); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 698, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_3 = __pyx_t_4;
+ __pyx_t_4 = 0;
+ goto __pyx_L14_bool_binop_done;
+ }
+ __pyx_t_4 = PyInt_FromSsize_t(__pyx_v_nslices); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 698, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_3 = __pyx_t_4;
+ __pyx_t_4 = 0;
+ __pyx_L14_bool_binop_done:;
+ __pyx_t_4 = PyList_AsTuple(__pyx_v_result); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 698, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __pyx_t_11 = PyTuple_New(2); if (unlikely(!__pyx_t_11)) __PYX_ERR(1, 698, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_11);
+ __Pyx_GIVEREF(__pyx_t_3);
+ PyTuple_SET_ITEM(__pyx_t_11, 0, __pyx_t_3);
+ __Pyx_GIVEREF(__pyx_t_4);
+ PyTuple_SET_ITEM(__pyx_t_11, 1, __pyx_t_4);
+ __pyx_t_3 = 0;
+ __pyx_t_4 = 0;
+ __pyx_r = ((PyObject*)__pyx_t_11);
+ __pyx_t_11 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":666
+ * return isinstance(o, memoryview)
+ *
+ * cdef tuple _unellipsify(object index, int ndim): # <<<<<<<<<<<<<<
+ * """
+ * Replace all ellipses with full slices and fill incomplete indices with
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_XDECREF(__pyx_t_7);
+ __Pyx_XDECREF(__pyx_t_11);
+ __Pyx_AddTraceback("View.MemoryView._unellipsify", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XDECREF(__pyx_v_tup);
+ __Pyx_XDECREF(__pyx_v_result);
+ __Pyx_XDECREF(__pyx_v_idx);
+ __Pyx_XDECREF(__pyx_v_item);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":700
+ * return have_slices or nslices, tuple(result)
+ *
+ * cdef assert_direct_dimensions(Py_ssize_t *suboffsets, int ndim): # <<<<<<<<<<<<<<
+ * for suboffset in suboffsets[:ndim]:
+ * if suboffset >= 0:
+ */
+
+static PyObject *assert_direct_dimensions(Py_ssize_t *__pyx_v_suboffsets, int __pyx_v_ndim) {
+ Py_ssize_t __pyx_v_suboffset;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ Py_ssize_t *__pyx_t_1;
+ Py_ssize_t *__pyx_t_2;
+ Py_ssize_t *__pyx_t_3;
+ int __pyx_t_4;
+ PyObject *__pyx_t_5 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("assert_direct_dimensions", 0);
+
+ /* "View.MemoryView":701
+ *
+ * cdef assert_direct_dimensions(Py_ssize_t *suboffsets, int ndim):
+ * for suboffset in suboffsets[:ndim]: # <<<<<<<<<<<<<<
+ * if suboffset >= 0:
+ * raise ValueError("Indirect dimensions not supported")
+ */
+ __pyx_t_2 = (__pyx_v_suboffsets + __pyx_v_ndim);
+ for (__pyx_t_3 = __pyx_v_suboffsets; __pyx_t_3 < __pyx_t_2; __pyx_t_3++) {
+ __pyx_t_1 = __pyx_t_3;
+ __pyx_v_suboffset = (__pyx_t_1[0]);
+
+ /* "View.MemoryView":702
+ * cdef assert_direct_dimensions(Py_ssize_t *suboffsets, int ndim):
+ * for suboffset in suboffsets[:ndim]:
+ * if suboffset >= 0: # <<<<<<<<<<<<<<
+ * raise ValueError("Indirect dimensions not supported")
+ *
+ */
+ __pyx_t_4 = ((__pyx_v_suboffset >= 0) != 0);
+ if (unlikely(__pyx_t_4)) {
+
+ /* "View.MemoryView":703
+ * for suboffset in suboffsets[:ndim]:
+ * if suboffset >= 0:
+ * raise ValueError("Indirect dimensions not supported") # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_t_5 = __Pyx_PyObject_Call(__pyx_builtin_ValueError, __pyx_tuple__20, NULL); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 703, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __Pyx_Raise(__pyx_t_5, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
+ __PYX_ERR(1, 703, __pyx_L1_error)
+
+ /* "View.MemoryView":702
+ * cdef assert_direct_dimensions(Py_ssize_t *suboffsets, int ndim):
+ * for suboffset in suboffsets[:ndim]:
+ * if suboffset >= 0: # <<<<<<<<<<<<<<
+ * raise ValueError("Indirect dimensions not supported")
+ *
+ */
+ }
+ }
+
+ /* "View.MemoryView":700
+ * return have_slices or nslices, tuple(result)
+ *
+ * cdef assert_direct_dimensions(Py_ssize_t *suboffsets, int ndim): # <<<<<<<<<<<<<<
+ * for suboffset in suboffsets[:ndim]:
+ * if suboffset >= 0:
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_AddTraceback("View.MemoryView.assert_direct_dimensions", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":710
+ *
+ * @cname('__pyx_memview_slice')
+ * cdef memoryview memview_slice(memoryview memview, object indices): # <<<<<<<<<<<<<<
+ * cdef int new_ndim = 0, suboffset_dim = -1, dim
+ * cdef bint negative_step
+ */
+
+static struct __pyx_memoryview_obj *__pyx_memview_slice(struct __pyx_memoryview_obj *__pyx_v_memview, PyObject *__pyx_v_indices) {
+ int __pyx_v_new_ndim;
+ int __pyx_v_suboffset_dim;
+ int __pyx_v_dim;
+ __Pyx_memviewslice __pyx_v_src;
+ __Pyx_memviewslice __pyx_v_dst;
+ __Pyx_memviewslice *__pyx_v_p_src;
+ struct __pyx_memoryviewslice_obj *__pyx_v_memviewsliceobj = 0;
+ __Pyx_memviewslice *__pyx_v_p_dst;
+ int *__pyx_v_p_suboffset_dim;
+ Py_ssize_t __pyx_v_start;
+ Py_ssize_t __pyx_v_stop;
+ Py_ssize_t __pyx_v_step;
+ int __pyx_v_have_start;
+ int __pyx_v_have_stop;
+ int __pyx_v_have_step;
+ PyObject *__pyx_v_index = NULL;
+ struct __pyx_memoryview_obj *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ struct __pyx_memoryview_obj *__pyx_t_4;
+ char *__pyx_t_5;
+ int __pyx_t_6;
+ Py_ssize_t __pyx_t_7;
+ PyObject *(*__pyx_t_8)(PyObject *);
+ PyObject *__pyx_t_9 = NULL;
+ Py_ssize_t __pyx_t_10;
+ int __pyx_t_11;
+ Py_ssize_t __pyx_t_12;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("memview_slice", 0);
+
+ /* "View.MemoryView":711
+ * @cname('__pyx_memview_slice')
+ * cdef memoryview memview_slice(memoryview memview, object indices):
+ * cdef int new_ndim = 0, suboffset_dim = -1, dim # <<<<<<<<<<<<<<
+ * cdef bint negative_step
+ * cdef __Pyx_memviewslice src, dst
+ */
+ __pyx_v_new_ndim = 0;
+ __pyx_v_suboffset_dim = -1;
+
+ /* "View.MemoryView":718
+ *
+ *
+ * memset(&dst, 0, sizeof(dst)) # <<<<<<<<<<<<<<
+ *
+ * cdef _memoryviewslice memviewsliceobj
+ */
+ (void)(memset((&__pyx_v_dst), 0, (sizeof(__pyx_v_dst))));
+
+ /* "View.MemoryView":722
+ * cdef _memoryviewslice memviewsliceobj
+ *
+ * assert memview.view.ndim > 0 # <<<<<<<<<<<<<<
+ *
+ * if isinstance(memview, _memoryviewslice):
+ */
+ #ifndef CYTHON_WITHOUT_ASSERTIONS
+ if (unlikely(!Py_OptimizeFlag)) {
+ if (unlikely(!((__pyx_v_memview->view.ndim > 0) != 0))) {
+ PyErr_SetNone(PyExc_AssertionError);
+ __PYX_ERR(1, 722, __pyx_L1_error)
+ }
+ }
+ #endif
+
+ /* "View.MemoryView":724
+ * assert memview.view.ndim > 0
+ *
+ * if isinstance(memview, _memoryviewslice): # <<<<<<<<<<<<<<
+ * memviewsliceobj = memview
+ * p_src = &memviewsliceobj.from_slice
+ */
+ __pyx_t_1 = __Pyx_TypeCheck(((PyObject *)__pyx_v_memview), __pyx_memoryviewslice_type);
+ __pyx_t_2 = (__pyx_t_1 != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":725
+ *
+ * if isinstance(memview, _memoryviewslice):
+ * memviewsliceobj = memview # <<<<<<<<<<<<<<
+ * p_src = &memviewsliceobj.from_slice
+ * else:
+ */
+ if (!(likely(((((PyObject *)__pyx_v_memview)) == Py_None) || likely(__Pyx_TypeTest(((PyObject *)__pyx_v_memview), __pyx_memoryviewslice_type))))) __PYX_ERR(1, 725, __pyx_L1_error)
+ __pyx_t_3 = ((PyObject *)__pyx_v_memview);
+ __Pyx_INCREF(__pyx_t_3);
+ __pyx_v_memviewsliceobj = ((struct __pyx_memoryviewslice_obj *)__pyx_t_3);
+ __pyx_t_3 = 0;
+
+ /* "View.MemoryView":726
+ * if isinstance(memview, _memoryviewslice):
+ * memviewsliceobj = memview
+ * p_src = &memviewsliceobj.from_slice # <<<<<<<<<<<<<<
+ * else:
+ * slice_copy(memview, &src)
+ */
+ __pyx_v_p_src = (&__pyx_v_memviewsliceobj->from_slice);
+
+ /* "View.MemoryView":724
+ * assert memview.view.ndim > 0
+ *
+ * if isinstance(memview, _memoryviewslice): # <<<<<<<<<<<<<<
+ * memviewsliceobj = memview
+ * p_src = &memviewsliceobj.from_slice
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":728
+ * p_src = &memviewsliceobj.from_slice
+ * else:
+ * slice_copy(memview, &src) # <<<<<<<<<<<<<<
+ * p_src = &src
+ *
+ */
+ /*else*/ {
+ __pyx_memoryview_slice_copy(__pyx_v_memview, (&__pyx_v_src));
+
+ /* "View.MemoryView":729
+ * else:
+ * slice_copy(memview, &src)
+ * p_src = &src # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_v_p_src = (&__pyx_v_src);
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":735
+ *
+ *
+ * dst.memview = p_src.memview # <<<<<<<<<<<<<<
+ * dst.data = p_src.data
+ *
+ */
+ __pyx_t_4 = __pyx_v_p_src->memview;
+ __pyx_v_dst.memview = __pyx_t_4;
+
+ /* "View.MemoryView":736
+ *
+ * dst.memview = p_src.memview
+ * dst.data = p_src.data # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_t_5 = __pyx_v_p_src->data;
+ __pyx_v_dst.data = __pyx_t_5;
+
+ /* "View.MemoryView":741
+ *
+ *
+ * cdef __Pyx_memviewslice *p_dst = &dst # <<<<<<<<<<<<<<
+ * cdef int *p_suboffset_dim = &suboffset_dim
+ * cdef Py_ssize_t start, stop, step
+ */
+ __pyx_v_p_dst = (&__pyx_v_dst);
+
+ /* "View.MemoryView":742
+ *
+ * cdef __Pyx_memviewslice *p_dst = &dst
+ * cdef int *p_suboffset_dim = &suboffset_dim # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t start, stop, step
+ * cdef bint have_start, have_stop, have_step
+ */
+ __pyx_v_p_suboffset_dim = (&__pyx_v_suboffset_dim);
+
+ /* "View.MemoryView":746
+ * cdef bint have_start, have_stop, have_step
+ *
+ * for dim, index in enumerate(indices): # <<<<<<<<<<<<<<
+ * if PyIndex_Check(index):
+ * slice_memviewslice(
+ */
+ __pyx_t_6 = 0;
+ if (likely(PyList_CheckExact(__pyx_v_indices)) || PyTuple_CheckExact(__pyx_v_indices)) {
+ __pyx_t_3 = __pyx_v_indices; __Pyx_INCREF(__pyx_t_3); __pyx_t_7 = 0;
+ __pyx_t_8 = NULL;
+ } else {
+ __pyx_t_7 = -1; __pyx_t_3 = PyObject_GetIter(__pyx_v_indices); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 746, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_t_8 = Py_TYPE(__pyx_t_3)->tp_iternext; if (unlikely(!__pyx_t_8)) __PYX_ERR(1, 746, __pyx_L1_error)
+ }
+ for (;;) {
+ if (likely(!__pyx_t_8)) {
+ if (likely(PyList_CheckExact(__pyx_t_3))) {
+ if (__pyx_t_7 >= PyList_GET_SIZE(__pyx_t_3)) break;
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_9 = PyList_GET_ITEM(__pyx_t_3, __pyx_t_7); __Pyx_INCREF(__pyx_t_9); __pyx_t_7++; if (unlikely(0 < 0)) __PYX_ERR(1, 746, __pyx_L1_error)
+ #else
+ __pyx_t_9 = PySequence_ITEM(__pyx_t_3, __pyx_t_7); __pyx_t_7++; if (unlikely(!__pyx_t_9)) __PYX_ERR(1, 746, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_9);
+ #endif
+ } else {
+ if (__pyx_t_7 >= PyTuple_GET_SIZE(__pyx_t_3)) break;
+ #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
+ __pyx_t_9 = PyTuple_GET_ITEM(__pyx_t_3, __pyx_t_7); __Pyx_INCREF(__pyx_t_9); __pyx_t_7++; if (unlikely(0 < 0)) __PYX_ERR(1, 746, __pyx_L1_error)
+ #else
+ __pyx_t_9 = PySequence_ITEM(__pyx_t_3, __pyx_t_7); __pyx_t_7++; if (unlikely(!__pyx_t_9)) __PYX_ERR(1, 746, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_9);
+ #endif
+ }
+ } else {
+ __pyx_t_9 = __pyx_t_8(__pyx_t_3);
+ if (unlikely(!__pyx_t_9)) {
+ PyObject* exc_type = PyErr_Occurred();
+ if (exc_type) {
+ if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear();
+ else __PYX_ERR(1, 746, __pyx_L1_error)
+ }
+ break;
+ }
+ __Pyx_GOTREF(__pyx_t_9);
+ }
+ __Pyx_XDECREF_SET(__pyx_v_index, __pyx_t_9);
+ __pyx_t_9 = 0;
+ __pyx_v_dim = __pyx_t_6;
+ __pyx_t_6 = (__pyx_t_6 + 1);
+
+ /* "View.MemoryView":747
+ *
+ * for dim, index in enumerate(indices):
+ * if PyIndex_Check(index): # <<<<<<<<<<<<<<
+ * slice_memviewslice(
+ * p_dst, p_src.shape[dim], p_src.strides[dim], p_src.suboffsets[dim],
+ */
+ __pyx_t_2 = (PyIndex_Check(__pyx_v_index) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":751
+ * p_dst, p_src.shape[dim], p_src.strides[dim], p_src.suboffsets[dim],
+ * dim, new_ndim, p_suboffset_dim,
+ * index, 0, 0, # start, stop, step # <<<<<<<<<<<<<<
+ * 0, 0, 0, # have_{start,stop,step}
+ * False)
+ */
+ __pyx_t_10 = __Pyx_PyIndex_AsSsize_t(__pyx_v_index); if (unlikely((__pyx_t_10 == (Py_ssize_t)-1) && PyErr_Occurred())) __PYX_ERR(1, 751, __pyx_L1_error)
+
+ /* "View.MemoryView":748
+ * for dim, index in enumerate(indices):
+ * if PyIndex_Check(index):
+ * slice_memviewslice( # <<<<<<<<<<<<<<
+ * p_dst, p_src.shape[dim], p_src.strides[dim], p_src.suboffsets[dim],
+ * dim, new_ndim, p_suboffset_dim,
+ */
+ __pyx_t_11 = __pyx_memoryview_slice_memviewslice(__pyx_v_p_dst, (__pyx_v_p_src->shape[__pyx_v_dim]), (__pyx_v_p_src->strides[__pyx_v_dim]), (__pyx_v_p_src->suboffsets[__pyx_v_dim]), __pyx_v_dim, __pyx_v_new_ndim, __pyx_v_p_suboffset_dim, __pyx_t_10, 0, 0, 0, 0, 0, 0); if (unlikely(__pyx_t_11 == ((int)-1))) __PYX_ERR(1, 748, __pyx_L1_error)
+
+ /* "View.MemoryView":747
+ *
+ * for dim, index in enumerate(indices):
+ * if PyIndex_Check(index): # <<<<<<<<<<<<<<
+ * slice_memviewslice(
+ * p_dst, p_src.shape[dim], p_src.strides[dim], p_src.suboffsets[dim],
+ */
+ goto __pyx_L6;
+ }
+
+ /* "View.MemoryView":754
+ * 0, 0, 0, # have_{start,stop,step}
+ * False)
+ * elif index is None: # <<<<<<<<<<<<<<
+ * p_dst.shape[new_ndim] = 1
+ * p_dst.strides[new_ndim] = 0
+ */
+ __pyx_t_2 = (__pyx_v_index == Py_None);
+ __pyx_t_1 = (__pyx_t_2 != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":755
+ * False)
+ * elif index is None:
+ * p_dst.shape[new_ndim] = 1 # <<<<<<<<<<<<<<
+ * p_dst.strides[new_ndim] = 0
+ * p_dst.suboffsets[new_ndim] = -1
+ */
+ (__pyx_v_p_dst->shape[__pyx_v_new_ndim]) = 1;
+
+ /* "View.MemoryView":756
+ * elif index is None:
+ * p_dst.shape[new_ndim] = 1
+ * p_dst.strides[new_ndim] = 0 # <<<<<<<<<<<<<<
+ * p_dst.suboffsets[new_ndim] = -1
+ * new_ndim += 1
+ */
+ (__pyx_v_p_dst->strides[__pyx_v_new_ndim]) = 0;
+
+ /* "View.MemoryView":757
+ * p_dst.shape[new_ndim] = 1
+ * p_dst.strides[new_ndim] = 0
+ * p_dst.suboffsets[new_ndim] = -1 # <<<<<<<<<<<<<<
+ * new_ndim += 1
+ * else:
+ */
+ (__pyx_v_p_dst->suboffsets[__pyx_v_new_ndim]) = -1L;
+
+ /* "View.MemoryView":758
+ * p_dst.strides[new_ndim] = 0
+ * p_dst.suboffsets[new_ndim] = -1
+ * new_ndim += 1 # <<<<<<<<<<<<<<
+ * else:
+ * start = index.start or 0
+ */
+ __pyx_v_new_ndim = (__pyx_v_new_ndim + 1);
+
+ /* "View.MemoryView":754
+ * 0, 0, 0, # have_{start,stop,step}
+ * False)
+ * elif index is None: # <<<<<<<<<<<<<<
+ * p_dst.shape[new_ndim] = 1
+ * p_dst.strides[new_ndim] = 0
+ */
+ goto __pyx_L6;
+ }
+
+ /* "View.MemoryView":760
+ * new_ndim += 1
+ * else:
+ * start = index.start or 0 # <<<<<<<<<<<<<<
+ * stop = index.stop or 0
+ * step = index.step or 0
+ */
+ /*else*/ {
+ __pyx_t_9 = __Pyx_PyObject_GetAttrStr(__pyx_v_index, __pyx_n_s_start); if (unlikely(!__pyx_t_9)) __PYX_ERR(1, 760, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_9);
+ __pyx_t_1 = __Pyx_PyObject_IsTrue(__pyx_t_9); if (unlikely(__pyx_t_1 < 0)) __PYX_ERR(1, 760, __pyx_L1_error)
+ if (!__pyx_t_1) {
+ __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
+ } else {
+ __pyx_t_12 = __Pyx_PyIndex_AsSsize_t(__pyx_t_9); if (unlikely((__pyx_t_12 == (Py_ssize_t)-1) && PyErr_Occurred())) __PYX_ERR(1, 760, __pyx_L1_error)
+ __pyx_t_10 = __pyx_t_12;
+ __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
+ goto __pyx_L7_bool_binop_done;
+ }
+ __pyx_t_10 = 0;
+ __pyx_L7_bool_binop_done:;
+ __pyx_v_start = __pyx_t_10;
+
+ /* "View.MemoryView":761
+ * else:
+ * start = index.start or 0
+ * stop = index.stop or 0 # <<<<<<<<<<<<<<
+ * step = index.step or 0
+ *
+ */
+ __pyx_t_9 = __Pyx_PyObject_GetAttrStr(__pyx_v_index, __pyx_n_s_stop); if (unlikely(!__pyx_t_9)) __PYX_ERR(1, 761, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_9);
+ __pyx_t_1 = __Pyx_PyObject_IsTrue(__pyx_t_9); if (unlikely(__pyx_t_1 < 0)) __PYX_ERR(1, 761, __pyx_L1_error)
+ if (!__pyx_t_1) {
+ __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
+ } else {
+ __pyx_t_12 = __Pyx_PyIndex_AsSsize_t(__pyx_t_9); if (unlikely((__pyx_t_12 == (Py_ssize_t)-1) && PyErr_Occurred())) __PYX_ERR(1, 761, __pyx_L1_error)
+ __pyx_t_10 = __pyx_t_12;
+ __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
+ goto __pyx_L9_bool_binop_done;
+ }
+ __pyx_t_10 = 0;
+ __pyx_L9_bool_binop_done:;
+ __pyx_v_stop = __pyx_t_10;
+
+ /* "View.MemoryView":762
+ * start = index.start or 0
+ * stop = index.stop or 0
+ * step = index.step or 0 # <<<<<<<<<<<<<<
+ *
+ * have_start = index.start is not None
+ */
+ __pyx_t_9 = __Pyx_PyObject_GetAttrStr(__pyx_v_index, __pyx_n_s_step); if (unlikely(!__pyx_t_9)) __PYX_ERR(1, 762, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_9);
+ __pyx_t_1 = __Pyx_PyObject_IsTrue(__pyx_t_9); if (unlikely(__pyx_t_1 < 0)) __PYX_ERR(1, 762, __pyx_L1_error)
+ if (!__pyx_t_1) {
+ __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
+ } else {
+ __pyx_t_12 = __Pyx_PyIndex_AsSsize_t(__pyx_t_9); if (unlikely((__pyx_t_12 == (Py_ssize_t)-1) && PyErr_Occurred())) __PYX_ERR(1, 762, __pyx_L1_error)
+ __pyx_t_10 = __pyx_t_12;
+ __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
+ goto __pyx_L11_bool_binop_done;
+ }
+ __pyx_t_10 = 0;
+ __pyx_L11_bool_binop_done:;
+ __pyx_v_step = __pyx_t_10;
+
+ /* "View.MemoryView":764
+ * step = index.step or 0
+ *
+ * have_start = index.start is not None # <<<<<<<<<<<<<<
+ * have_stop = index.stop is not None
+ * have_step = index.step is not None
+ */
+ __pyx_t_9 = __Pyx_PyObject_GetAttrStr(__pyx_v_index, __pyx_n_s_start); if (unlikely(!__pyx_t_9)) __PYX_ERR(1, 764, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_9);
+ __pyx_t_1 = (__pyx_t_9 != Py_None);
+ __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
+ __pyx_v_have_start = __pyx_t_1;
+
+ /* "View.MemoryView":765
+ *
+ * have_start = index.start is not None
+ * have_stop = index.stop is not None # <<<<<<<<<<<<<<
+ * have_step = index.step is not None
+ *
+ */
+ __pyx_t_9 = __Pyx_PyObject_GetAttrStr(__pyx_v_index, __pyx_n_s_stop); if (unlikely(!__pyx_t_9)) __PYX_ERR(1, 765, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_9);
+ __pyx_t_1 = (__pyx_t_9 != Py_None);
+ __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
+ __pyx_v_have_stop = __pyx_t_1;
+
+ /* "View.MemoryView":766
+ * have_start = index.start is not None
+ * have_stop = index.stop is not None
+ * have_step = index.step is not None # <<<<<<<<<<<<<<
+ *
+ * slice_memviewslice(
+ */
+ __pyx_t_9 = __Pyx_PyObject_GetAttrStr(__pyx_v_index, __pyx_n_s_step); if (unlikely(!__pyx_t_9)) __PYX_ERR(1, 766, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_9);
+ __pyx_t_1 = (__pyx_t_9 != Py_None);
+ __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
+ __pyx_v_have_step = __pyx_t_1;
+
+ /* "View.MemoryView":768
+ * have_step = index.step is not None
+ *
+ * slice_memviewslice( # <<<<<<<<<<<<<<
+ * p_dst, p_src.shape[dim], p_src.strides[dim], p_src.suboffsets[dim],
+ * dim, new_ndim, p_suboffset_dim,
+ */
+ __pyx_t_11 = __pyx_memoryview_slice_memviewslice(__pyx_v_p_dst, (__pyx_v_p_src->shape[__pyx_v_dim]), (__pyx_v_p_src->strides[__pyx_v_dim]), (__pyx_v_p_src->suboffsets[__pyx_v_dim]), __pyx_v_dim, __pyx_v_new_ndim, __pyx_v_p_suboffset_dim, __pyx_v_start, __pyx_v_stop, __pyx_v_step, __pyx_v_have_start, __pyx_v_have_stop, __pyx_v_have_step, 1); if (unlikely(__pyx_t_11 == ((int)-1))) __PYX_ERR(1, 768, __pyx_L1_error)
+
+ /* "View.MemoryView":774
+ * have_start, have_stop, have_step,
+ * True)
+ * new_ndim += 1 # <<<<<<<<<<<<<<
+ *
+ * if isinstance(memview, _memoryviewslice):
+ */
+ __pyx_v_new_ndim = (__pyx_v_new_ndim + 1);
+ }
+ __pyx_L6:;
+
+ /* "View.MemoryView":746
+ * cdef bint have_start, have_stop, have_step
+ *
+ * for dim, index in enumerate(indices): # <<<<<<<<<<<<<<
+ * if PyIndex_Check(index):
+ * slice_memviewslice(
+ */
+ }
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+
+ /* "View.MemoryView":776
+ * new_ndim += 1
+ *
+ * if isinstance(memview, _memoryviewslice): # <<<<<<<<<<<<<<
+ * return memoryview_fromslice(dst, new_ndim,
+ * memviewsliceobj.to_object_func,
+ */
+ __pyx_t_1 = __Pyx_TypeCheck(((PyObject *)__pyx_v_memview), __pyx_memoryviewslice_type);
+ __pyx_t_2 = (__pyx_t_1 != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":777
+ *
+ * if isinstance(memview, _memoryviewslice):
+ * return memoryview_fromslice(dst, new_ndim, # <<<<<<<<<<<<<<
+ * memviewsliceobj.to_object_func,
+ * memviewsliceobj.to_dtype_func,
+ */
+ __Pyx_XDECREF(((PyObject *)__pyx_r));
+
+ /* "View.MemoryView":778
+ * if isinstance(memview, _memoryviewslice):
+ * return memoryview_fromslice(dst, new_ndim,
+ * memviewsliceobj.to_object_func, # <<<<<<<<<<<<<<
+ * memviewsliceobj.to_dtype_func,
+ * memview.dtype_is_object)
+ */
+ if (unlikely(!__pyx_v_memviewsliceobj)) { __Pyx_RaiseUnboundLocalError("memviewsliceobj"); __PYX_ERR(1, 778, __pyx_L1_error) }
+
+ /* "View.MemoryView":779
+ * return memoryview_fromslice(dst, new_ndim,
+ * memviewsliceobj.to_object_func,
+ * memviewsliceobj.to_dtype_func, # <<<<<<<<<<<<<<
+ * memview.dtype_is_object)
+ * else:
+ */
+ if (unlikely(!__pyx_v_memviewsliceobj)) { __Pyx_RaiseUnboundLocalError("memviewsliceobj"); __PYX_ERR(1, 779, __pyx_L1_error) }
+
+ /* "View.MemoryView":777
+ *
+ * if isinstance(memview, _memoryviewslice):
+ * return memoryview_fromslice(dst, new_ndim, # <<<<<<<<<<<<<<
+ * memviewsliceobj.to_object_func,
+ * memviewsliceobj.to_dtype_func,
+ */
+ __pyx_t_3 = __pyx_memoryview_fromslice(__pyx_v_dst, __pyx_v_new_ndim, __pyx_v_memviewsliceobj->to_object_func, __pyx_v_memviewsliceobj->to_dtype_func, __pyx_v_memview->dtype_is_object); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 777, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ if (!(likely(((__pyx_t_3) == Py_None) || likely(__Pyx_TypeTest(__pyx_t_3, __pyx_memoryview_type))))) __PYX_ERR(1, 777, __pyx_L1_error)
+ __pyx_r = ((struct __pyx_memoryview_obj *)__pyx_t_3);
+ __pyx_t_3 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":776
+ * new_ndim += 1
+ *
+ * if isinstance(memview, _memoryviewslice): # <<<<<<<<<<<<<<
+ * return memoryview_fromslice(dst, new_ndim,
+ * memviewsliceobj.to_object_func,
+ */
+ }
+
+ /* "View.MemoryView":782
+ * memview.dtype_is_object)
+ * else:
+ * return memoryview_fromslice(dst, new_ndim, NULL, NULL, # <<<<<<<<<<<<<<
+ * memview.dtype_is_object)
+ *
+ */
+ /*else*/ {
+ __Pyx_XDECREF(((PyObject *)__pyx_r));
+
+ /* "View.MemoryView":783
+ * else:
+ * return memoryview_fromslice(dst, new_ndim, NULL, NULL,
+ * memview.dtype_is_object) # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_t_3 = __pyx_memoryview_fromslice(__pyx_v_dst, __pyx_v_new_ndim, NULL, NULL, __pyx_v_memview->dtype_is_object); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 782, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+
+ /* "View.MemoryView":782
+ * memview.dtype_is_object)
+ * else:
+ * return memoryview_fromslice(dst, new_ndim, NULL, NULL, # <<<<<<<<<<<<<<
+ * memview.dtype_is_object)
+ *
+ */
+ if (!(likely(((__pyx_t_3) == Py_None) || likely(__Pyx_TypeTest(__pyx_t_3, __pyx_memoryview_type))))) __PYX_ERR(1, 782, __pyx_L1_error)
+ __pyx_r = ((struct __pyx_memoryview_obj *)__pyx_t_3);
+ __pyx_t_3 = 0;
+ goto __pyx_L0;
+ }
+
+ /* "View.MemoryView":710
+ *
+ * @cname('__pyx_memview_slice')
+ * cdef memoryview memview_slice(memoryview memview, object indices): # <<<<<<<<<<<<<<
+ * cdef int new_ndim = 0, suboffset_dim = -1, dim
+ * cdef bint negative_step
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_XDECREF(__pyx_t_9);
+ __Pyx_AddTraceback("View.MemoryView.memview_slice", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XDECREF((PyObject *)__pyx_v_memviewsliceobj);
+ __Pyx_XDECREF(__pyx_v_index);
+ __Pyx_XGIVEREF((PyObject *)__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":807
+ *
+ * @cname('__pyx_memoryview_slice_memviewslice')
+ * cdef int slice_memviewslice( # <<<<<<<<<<<<<<
+ * __Pyx_memviewslice *dst,
+ * Py_ssize_t shape, Py_ssize_t stride, Py_ssize_t suboffset,
+ */
+
+static int __pyx_memoryview_slice_memviewslice(__Pyx_memviewslice *__pyx_v_dst, Py_ssize_t __pyx_v_shape, Py_ssize_t __pyx_v_stride, Py_ssize_t __pyx_v_suboffset, int __pyx_v_dim, int __pyx_v_new_ndim, int *__pyx_v_suboffset_dim, Py_ssize_t __pyx_v_start, Py_ssize_t __pyx_v_stop, Py_ssize_t __pyx_v_step, int __pyx_v_have_start, int __pyx_v_have_stop, int __pyx_v_have_step, int __pyx_v_is_slice) {
+ Py_ssize_t __pyx_v_new_shape;
+ int __pyx_v_negative_step;
+ int __pyx_r;
+ int __pyx_t_1;
+ int __pyx_t_2;
+ int __pyx_t_3;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+
+ /* "View.MemoryView":827
+ * cdef bint negative_step
+ *
+ * if not is_slice: # <<<<<<<<<<<<<<
+ *
+ * if start < 0:
+ */
+ __pyx_t_1 = ((!(__pyx_v_is_slice != 0)) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":829
+ * if not is_slice:
+ *
+ * if start < 0: # <<<<<<<<<<<<<<
+ * start += shape
+ * if not 0 <= start < shape:
+ */
+ __pyx_t_1 = ((__pyx_v_start < 0) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":830
+ *
+ * if start < 0:
+ * start += shape # <<<<<<<<<<<<<<
+ * if not 0 <= start < shape:
+ * _err_dim(IndexError, "Index out of bounds (axis %d)", dim)
+ */
+ __pyx_v_start = (__pyx_v_start + __pyx_v_shape);
+
+ /* "View.MemoryView":829
+ * if not is_slice:
+ *
+ * if start < 0: # <<<<<<<<<<<<<<
+ * start += shape
+ * if not 0 <= start < shape:
+ */
+ }
+
+ /* "View.MemoryView":831
+ * if start < 0:
+ * start += shape
+ * if not 0 <= start < shape: # <<<<<<<<<<<<<<
+ * _err_dim(IndexError, "Index out of bounds (axis %d)", dim)
+ * else:
+ */
+ __pyx_t_1 = (0 <= __pyx_v_start);
+ if (__pyx_t_1) {
+ __pyx_t_1 = (__pyx_v_start < __pyx_v_shape);
+ }
+ __pyx_t_2 = ((!(__pyx_t_1 != 0)) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":832
+ * start += shape
+ * if not 0 <= start < shape:
+ * _err_dim(IndexError, "Index out of bounds (axis %d)", dim) # <<<<<<<<<<<<<<
+ * else:
+ *
+ */
+ __pyx_t_3 = __pyx_memoryview_err_dim(__pyx_builtin_IndexError, ((char *)"Index out of bounds (axis %d)"), __pyx_v_dim); if (unlikely(__pyx_t_3 == ((int)-1))) __PYX_ERR(1, 832, __pyx_L1_error)
+
+ /* "View.MemoryView":831
+ * if start < 0:
+ * start += shape
+ * if not 0 <= start < shape: # <<<<<<<<<<<<<<
+ * _err_dim(IndexError, "Index out of bounds (axis %d)", dim)
+ * else:
+ */
+ }
+
+ /* "View.MemoryView":827
+ * cdef bint negative_step
+ *
+ * if not is_slice: # <<<<<<<<<<<<<<
+ *
+ * if start < 0:
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":835
+ * else:
+ *
+ * negative_step = have_step != 0 and step < 0 # <<<<<<<<<<<<<<
+ *
+ * if have_step and step == 0:
+ */
+ /*else*/ {
+ __pyx_t_1 = ((__pyx_v_have_step != 0) != 0);
+ if (__pyx_t_1) {
+ } else {
+ __pyx_t_2 = __pyx_t_1;
+ goto __pyx_L6_bool_binop_done;
+ }
+ __pyx_t_1 = ((__pyx_v_step < 0) != 0);
+ __pyx_t_2 = __pyx_t_1;
+ __pyx_L6_bool_binop_done:;
+ __pyx_v_negative_step = __pyx_t_2;
+
+ /* "View.MemoryView":837
+ * negative_step = have_step != 0 and step < 0
+ *
+ * if have_step and step == 0: # <<<<<<<<<<<<<<
+ * _err_dim(ValueError, "Step may not be zero (axis %d)", dim)
+ *
+ */
+ __pyx_t_1 = (__pyx_v_have_step != 0);
+ if (__pyx_t_1) {
+ } else {
+ __pyx_t_2 = __pyx_t_1;
+ goto __pyx_L9_bool_binop_done;
+ }
+ __pyx_t_1 = ((__pyx_v_step == 0) != 0);
+ __pyx_t_2 = __pyx_t_1;
+ __pyx_L9_bool_binop_done:;
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":838
+ *
+ * if have_step and step == 0:
+ * _err_dim(ValueError, "Step may not be zero (axis %d)", dim) # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_t_3 = __pyx_memoryview_err_dim(__pyx_builtin_ValueError, ((char *)"Step may not be zero (axis %d)"), __pyx_v_dim); if (unlikely(__pyx_t_3 == ((int)-1))) __PYX_ERR(1, 838, __pyx_L1_error)
+
+ /* "View.MemoryView":837
+ * negative_step = have_step != 0 and step < 0
+ *
+ * if have_step and step == 0: # <<<<<<<<<<<<<<
+ * _err_dim(ValueError, "Step may not be zero (axis %d)", dim)
+ *
+ */
+ }
+
+ /* "View.MemoryView":841
+ *
+ *
+ * if have_start: # <<<<<<<<<<<<<<
+ * if start < 0:
+ * start += shape
+ */
+ __pyx_t_2 = (__pyx_v_have_start != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":842
+ *
+ * if have_start:
+ * if start < 0: # <<<<<<<<<<<<<<
+ * start += shape
+ * if start < 0:
+ */
+ __pyx_t_2 = ((__pyx_v_start < 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":843
+ * if have_start:
+ * if start < 0:
+ * start += shape # <<<<<<<<<<<<<<
+ * if start < 0:
+ * start = 0
+ */
+ __pyx_v_start = (__pyx_v_start + __pyx_v_shape);
+
+ /* "View.MemoryView":844
+ * if start < 0:
+ * start += shape
+ * if start < 0: # <<<<<<<<<<<<<<
+ * start = 0
+ * elif start >= shape:
+ */
+ __pyx_t_2 = ((__pyx_v_start < 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":845
+ * start += shape
+ * if start < 0:
+ * start = 0 # <<<<<<<<<<<<<<
+ * elif start >= shape:
+ * if negative_step:
+ */
+ __pyx_v_start = 0;
+
+ /* "View.MemoryView":844
+ * if start < 0:
+ * start += shape
+ * if start < 0: # <<<<<<<<<<<<<<
+ * start = 0
+ * elif start >= shape:
+ */
+ }
+
+ /* "View.MemoryView":842
+ *
+ * if have_start:
+ * if start < 0: # <<<<<<<<<<<<<<
+ * start += shape
+ * if start < 0:
+ */
+ goto __pyx_L12;
+ }
+
+ /* "View.MemoryView":846
+ * if start < 0:
+ * start = 0
+ * elif start >= shape: # <<<<<<<<<<<<<<
+ * if negative_step:
+ * start = shape - 1
+ */
+ __pyx_t_2 = ((__pyx_v_start >= __pyx_v_shape) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":847
+ * start = 0
+ * elif start >= shape:
+ * if negative_step: # <<<<<<<<<<<<<<
+ * start = shape - 1
+ * else:
+ */
+ __pyx_t_2 = (__pyx_v_negative_step != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":848
+ * elif start >= shape:
+ * if negative_step:
+ * start = shape - 1 # <<<<<<<<<<<<<<
+ * else:
+ * start = shape
+ */
+ __pyx_v_start = (__pyx_v_shape - 1);
+
+ /* "View.MemoryView":847
+ * start = 0
+ * elif start >= shape:
+ * if negative_step: # <<<<<<<<<<<<<<
+ * start = shape - 1
+ * else:
+ */
+ goto __pyx_L14;
+ }
+
+ /* "View.MemoryView":850
+ * start = shape - 1
+ * else:
+ * start = shape # <<<<<<<<<<<<<<
+ * else:
+ * if negative_step:
+ */
+ /*else*/ {
+ __pyx_v_start = __pyx_v_shape;
+ }
+ __pyx_L14:;
+
+ /* "View.MemoryView":846
+ * if start < 0:
+ * start = 0
+ * elif start >= shape: # <<<<<<<<<<<<<<
+ * if negative_step:
+ * start = shape - 1
+ */
+ }
+ __pyx_L12:;
+
+ /* "View.MemoryView":841
+ *
+ *
+ * if have_start: # <<<<<<<<<<<<<<
+ * if start < 0:
+ * start += shape
+ */
+ goto __pyx_L11;
+ }
+
+ /* "View.MemoryView":852
+ * start = shape
+ * else:
+ * if negative_step: # <<<<<<<<<<<<<<
+ * start = shape - 1
+ * else:
+ */
+ /*else*/ {
+ __pyx_t_2 = (__pyx_v_negative_step != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":853
+ * else:
+ * if negative_step:
+ * start = shape - 1 # <<<<<<<<<<<<<<
+ * else:
+ * start = 0
+ */
+ __pyx_v_start = (__pyx_v_shape - 1);
+
+ /* "View.MemoryView":852
+ * start = shape
+ * else:
+ * if negative_step: # <<<<<<<<<<<<<<
+ * start = shape - 1
+ * else:
+ */
+ goto __pyx_L15;
+ }
+
+ /* "View.MemoryView":855
+ * start = shape - 1
+ * else:
+ * start = 0 # <<<<<<<<<<<<<<
+ *
+ * if have_stop:
+ */
+ /*else*/ {
+ __pyx_v_start = 0;
+ }
+ __pyx_L15:;
+ }
+ __pyx_L11:;
+
+ /* "View.MemoryView":857
+ * start = 0
+ *
+ * if have_stop: # <<<<<<<<<<<<<<
+ * if stop < 0:
+ * stop += shape
+ */
+ __pyx_t_2 = (__pyx_v_have_stop != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":858
+ *
+ * if have_stop:
+ * if stop < 0: # <<<<<<<<<<<<<<
+ * stop += shape
+ * if stop < 0:
+ */
+ __pyx_t_2 = ((__pyx_v_stop < 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":859
+ * if have_stop:
+ * if stop < 0:
+ * stop += shape # <<<<<<<<<<<<<<
+ * if stop < 0:
+ * stop = 0
+ */
+ __pyx_v_stop = (__pyx_v_stop + __pyx_v_shape);
+
+ /* "View.MemoryView":860
+ * if stop < 0:
+ * stop += shape
+ * if stop < 0: # <<<<<<<<<<<<<<
+ * stop = 0
+ * elif stop > shape:
+ */
+ __pyx_t_2 = ((__pyx_v_stop < 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":861
+ * stop += shape
+ * if stop < 0:
+ * stop = 0 # <<<<<<<<<<<<<<
+ * elif stop > shape:
+ * stop = shape
+ */
+ __pyx_v_stop = 0;
+
+ /* "View.MemoryView":860
+ * if stop < 0:
+ * stop += shape
+ * if stop < 0: # <<<<<<<<<<<<<<
+ * stop = 0
+ * elif stop > shape:
+ */
+ }
+
+ /* "View.MemoryView":858
+ *
+ * if have_stop:
+ * if stop < 0: # <<<<<<<<<<<<<<
+ * stop += shape
+ * if stop < 0:
+ */
+ goto __pyx_L17;
+ }
+
+ /* "View.MemoryView":862
+ * if stop < 0:
+ * stop = 0
+ * elif stop > shape: # <<<<<<<<<<<<<<
+ * stop = shape
+ * else:
+ */
+ __pyx_t_2 = ((__pyx_v_stop > __pyx_v_shape) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":863
+ * stop = 0
+ * elif stop > shape:
+ * stop = shape # <<<<<<<<<<<<<<
+ * else:
+ * if negative_step:
+ */
+ __pyx_v_stop = __pyx_v_shape;
+
+ /* "View.MemoryView":862
+ * if stop < 0:
+ * stop = 0
+ * elif stop > shape: # <<<<<<<<<<<<<<
+ * stop = shape
+ * else:
+ */
+ }
+ __pyx_L17:;
+
+ /* "View.MemoryView":857
+ * start = 0
+ *
+ * if have_stop: # <<<<<<<<<<<<<<
+ * if stop < 0:
+ * stop += shape
+ */
+ goto __pyx_L16;
+ }
+
+ /* "View.MemoryView":865
+ * stop = shape
+ * else:
+ * if negative_step: # <<<<<<<<<<<<<<
+ * stop = -1
+ * else:
+ */
+ /*else*/ {
+ __pyx_t_2 = (__pyx_v_negative_step != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":866
+ * else:
+ * if negative_step:
+ * stop = -1 # <<<<<<<<<<<<<<
+ * else:
+ * stop = shape
+ */
+ __pyx_v_stop = -1L;
+
+ /* "View.MemoryView":865
+ * stop = shape
+ * else:
+ * if negative_step: # <<<<<<<<<<<<<<
+ * stop = -1
+ * else:
+ */
+ goto __pyx_L19;
+ }
+
+ /* "View.MemoryView":868
+ * stop = -1
+ * else:
+ * stop = shape # <<<<<<<<<<<<<<
+ *
+ * if not have_step:
+ */
+ /*else*/ {
+ __pyx_v_stop = __pyx_v_shape;
+ }
+ __pyx_L19:;
+ }
+ __pyx_L16:;
+
+ /* "View.MemoryView":870
+ * stop = shape
+ *
+ * if not have_step: # <<<<<<<<<<<<<<
+ * step = 1
+ *
+ */
+ __pyx_t_2 = ((!(__pyx_v_have_step != 0)) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":871
+ *
+ * if not have_step:
+ * step = 1 # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_v_step = 1;
+
+ /* "View.MemoryView":870
+ * stop = shape
+ *
+ * if not have_step: # <<<<<<<<<<<<<<
+ * step = 1
+ *
+ */
+ }
+
+ /* "View.MemoryView":875
+ *
+ * with cython.cdivision(True):
+ * new_shape = (stop - start) // step # <<<<<<<<<<<<<<
+ *
+ * if (stop - start) - step * new_shape:
+ */
+ __pyx_v_new_shape = ((__pyx_v_stop - __pyx_v_start) / __pyx_v_step);
+
+ /* "View.MemoryView":877
+ * new_shape = (stop - start) // step
+ *
+ * if (stop - start) - step * new_shape: # <<<<<<<<<<<<<<
+ * new_shape += 1
+ *
+ */
+ __pyx_t_2 = (((__pyx_v_stop - __pyx_v_start) - (__pyx_v_step * __pyx_v_new_shape)) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":878
+ *
+ * if (stop - start) - step * new_shape:
+ * new_shape += 1 # <<<<<<<<<<<<<<
+ *
+ * if new_shape < 0:
+ */
+ __pyx_v_new_shape = (__pyx_v_new_shape + 1);
+
+ /* "View.MemoryView":877
+ * new_shape = (stop - start) // step
+ *
+ * if (stop - start) - step * new_shape: # <<<<<<<<<<<<<<
+ * new_shape += 1
+ *
+ */
+ }
+
+ /* "View.MemoryView":880
+ * new_shape += 1
+ *
+ * if new_shape < 0: # <<<<<<<<<<<<<<
+ * new_shape = 0
+ *
+ */
+ __pyx_t_2 = ((__pyx_v_new_shape < 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":881
+ *
+ * if new_shape < 0:
+ * new_shape = 0 # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_v_new_shape = 0;
+
+ /* "View.MemoryView":880
+ * new_shape += 1
+ *
+ * if new_shape < 0: # <<<<<<<<<<<<<<
+ * new_shape = 0
+ *
+ */
+ }
+
+ /* "View.MemoryView":884
+ *
+ *
+ * dst.strides[new_ndim] = stride * step # <<<<<<<<<<<<<<
+ * dst.shape[new_ndim] = new_shape
+ * dst.suboffsets[new_ndim] = suboffset
+ */
+ (__pyx_v_dst->strides[__pyx_v_new_ndim]) = (__pyx_v_stride * __pyx_v_step);
+
+ /* "View.MemoryView":885
+ *
+ * dst.strides[new_ndim] = stride * step
+ * dst.shape[new_ndim] = new_shape # <<<<<<<<<<<<<<
+ * dst.suboffsets[new_ndim] = suboffset
+ *
+ */
+ (__pyx_v_dst->shape[__pyx_v_new_ndim]) = __pyx_v_new_shape;
+
+ /* "View.MemoryView":886
+ * dst.strides[new_ndim] = stride * step
+ * dst.shape[new_ndim] = new_shape
+ * dst.suboffsets[new_ndim] = suboffset # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ (__pyx_v_dst->suboffsets[__pyx_v_new_ndim]) = __pyx_v_suboffset;
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":889
+ *
+ *
+ * if suboffset_dim[0] < 0: # <<<<<<<<<<<<<<
+ * dst.data += start * stride
+ * else:
+ */
+ __pyx_t_2 = (((__pyx_v_suboffset_dim[0]) < 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":890
+ *
+ * if suboffset_dim[0] < 0:
+ * dst.data += start * stride # <<<<<<<<<<<<<<
+ * else:
+ * dst.suboffsets[suboffset_dim[0]] += start * stride
+ */
+ __pyx_v_dst->data = (__pyx_v_dst->data + (__pyx_v_start * __pyx_v_stride));
+
+ /* "View.MemoryView":889
+ *
+ *
+ * if suboffset_dim[0] < 0: # <<<<<<<<<<<<<<
+ * dst.data += start * stride
+ * else:
+ */
+ goto __pyx_L23;
+ }
+
+ /* "View.MemoryView":892
+ * dst.data += start * stride
+ * else:
+ * dst.suboffsets[suboffset_dim[0]] += start * stride # <<<<<<<<<<<<<<
+ *
+ * if suboffset >= 0:
+ */
+ /*else*/ {
+ __pyx_t_3 = (__pyx_v_suboffset_dim[0]);
+ (__pyx_v_dst->suboffsets[__pyx_t_3]) = ((__pyx_v_dst->suboffsets[__pyx_t_3]) + (__pyx_v_start * __pyx_v_stride));
+ }
+ __pyx_L23:;
+
+ /* "View.MemoryView":894
+ * dst.suboffsets[suboffset_dim[0]] += start * stride
+ *
+ * if suboffset >= 0: # <<<<<<<<<<<<<<
+ * if not is_slice:
+ * if new_ndim == 0:
+ */
+ __pyx_t_2 = ((__pyx_v_suboffset >= 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":895
+ *
+ * if suboffset >= 0:
+ * if not is_slice: # <<<<<<<<<<<<<<
+ * if new_ndim == 0:
+ * dst.data = ( dst.data)[0] + suboffset
+ */
+ __pyx_t_2 = ((!(__pyx_v_is_slice != 0)) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":896
+ * if suboffset >= 0:
+ * if not is_slice:
+ * if new_ndim == 0: # <<<<<<<<<<<<<<
+ * dst.data = ( dst.data)[0] + suboffset
+ * else:
+ */
+ __pyx_t_2 = ((__pyx_v_new_ndim == 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":897
+ * if not is_slice:
+ * if new_ndim == 0:
+ * dst.data = ( dst.data)[0] + suboffset # <<<<<<<<<<<<<<
+ * else:
+ * _err_dim(IndexError, "All dimensions preceding dimension %d "
+ */
+ __pyx_v_dst->data = ((((char **)__pyx_v_dst->data)[0]) + __pyx_v_suboffset);
+
+ /* "View.MemoryView":896
+ * if suboffset >= 0:
+ * if not is_slice:
+ * if new_ndim == 0: # <<<<<<<<<<<<<<
+ * dst.data = ( dst.data)[0] + suboffset
+ * else:
+ */
+ goto __pyx_L26;
+ }
+
+ /* "View.MemoryView":899
+ * dst.data = ( dst.data)[0] + suboffset
+ * else:
+ * _err_dim(IndexError, "All dimensions preceding dimension %d " # <<<<<<<<<<<<<<
+ * "must be indexed and not sliced", dim)
+ * else:
+ */
+ /*else*/ {
+
+ /* "View.MemoryView":900
+ * else:
+ * _err_dim(IndexError, "All dimensions preceding dimension %d "
+ * "must be indexed and not sliced", dim) # <<<<<<<<<<<<<<
+ * else:
+ * suboffset_dim[0] = new_ndim
+ */
+ __pyx_t_3 = __pyx_memoryview_err_dim(__pyx_builtin_IndexError, ((char *)"All dimensions preceding dimension %d must be indexed and not sliced"), __pyx_v_dim); if (unlikely(__pyx_t_3 == ((int)-1))) __PYX_ERR(1, 899, __pyx_L1_error)
+ }
+ __pyx_L26:;
+
+ /* "View.MemoryView":895
+ *
+ * if suboffset >= 0:
+ * if not is_slice: # <<<<<<<<<<<<<<
+ * if new_ndim == 0:
+ * dst.data = ( dst.data)[0] + suboffset
+ */
+ goto __pyx_L25;
+ }
+
+ /* "View.MemoryView":902
+ * "must be indexed and not sliced", dim)
+ * else:
+ * suboffset_dim[0] = new_ndim # <<<<<<<<<<<<<<
+ *
+ * return 0
+ */
+ /*else*/ {
+ (__pyx_v_suboffset_dim[0]) = __pyx_v_new_ndim;
+ }
+ __pyx_L25:;
+
+ /* "View.MemoryView":894
+ * dst.suboffsets[suboffset_dim[0]] += start * stride
+ *
+ * if suboffset >= 0: # <<<<<<<<<<<<<<
+ * if not is_slice:
+ * if new_ndim == 0:
+ */
+ }
+
+ /* "View.MemoryView":904
+ * suboffset_dim[0] = new_ndim
+ *
+ * return 0 # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_r = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":807
+ *
+ * @cname('__pyx_memoryview_slice_memviewslice')
+ * cdef int slice_memviewslice( # <<<<<<<<<<<<<<
+ * __Pyx_memviewslice *dst,
+ * Py_ssize_t shape, Py_ssize_t stride, Py_ssize_t suboffset,
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ {
+ #ifdef WITH_THREAD
+ PyGILState_STATE __pyx_gilstate_save = __Pyx_PyGILState_Ensure();
+ #endif
+ __Pyx_AddTraceback("View.MemoryView.slice_memviewslice", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ #ifdef WITH_THREAD
+ __Pyx_PyGILState_Release(__pyx_gilstate_save);
+ #endif
+ }
+ __pyx_r = -1;
+ __pyx_L0:;
+ return __pyx_r;
+}
+
+/* "View.MemoryView":910
+ *
+ * @cname('__pyx_pybuffer_index')
+ * cdef char *pybuffer_index(Py_buffer *view, char *bufp, Py_ssize_t index, # <<<<<<<<<<<<<<
+ * Py_ssize_t dim) except NULL:
+ * cdef Py_ssize_t shape, stride, suboffset = -1
+ */
+
+static char *__pyx_pybuffer_index(Py_buffer *__pyx_v_view, char *__pyx_v_bufp, Py_ssize_t __pyx_v_index, Py_ssize_t __pyx_v_dim) {
+ Py_ssize_t __pyx_v_shape;
+ Py_ssize_t __pyx_v_stride;
+ Py_ssize_t __pyx_v_suboffset;
+ Py_ssize_t __pyx_v_itemsize;
+ char *__pyx_v_resultp;
+ char *__pyx_r;
+ __Pyx_RefNannyDeclarations
+ Py_ssize_t __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ PyObject *__pyx_t_4 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("pybuffer_index", 0);
+
+ /* "View.MemoryView":912
+ * cdef char *pybuffer_index(Py_buffer *view, char *bufp, Py_ssize_t index,
+ * Py_ssize_t dim) except NULL:
+ * cdef Py_ssize_t shape, stride, suboffset = -1 # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t itemsize = view.itemsize
+ * cdef char *resultp
+ */
+ __pyx_v_suboffset = -1L;
+
+ /* "View.MemoryView":913
+ * Py_ssize_t dim) except NULL:
+ * cdef Py_ssize_t shape, stride, suboffset = -1
+ * cdef Py_ssize_t itemsize = view.itemsize # <<<<<<<<<<<<<<
+ * cdef char *resultp
+ *
+ */
+ __pyx_t_1 = __pyx_v_view->itemsize;
+ __pyx_v_itemsize = __pyx_t_1;
+
+ /* "View.MemoryView":916
+ * cdef char *resultp
+ *
+ * if view.ndim == 0: # <<<<<<<<<<<<<<
+ * shape = view.len / itemsize
+ * stride = itemsize
+ */
+ __pyx_t_2 = ((__pyx_v_view->ndim == 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":917
+ *
+ * if view.ndim == 0:
+ * shape = view.len / itemsize # <<<<<<<<<<<<<<
+ * stride = itemsize
+ * else:
+ */
+ if (unlikely(__pyx_v_itemsize == 0)) {
+ PyErr_SetString(PyExc_ZeroDivisionError, "integer division or modulo by zero");
+ __PYX_ERR(1, 917, __pyx_L1_error)
+ }
+ else if (sizeof(Py_ssize_t) == sizeof(long) && (!(((Py_ssize_t)-1) > 0)) && unlikely(__pyx_v_itemsize == (Py_ssize_t)-1) && unlikely(UNARY_NEG_WOULD_OVERFLOW(__pyx_v_view->len))) {
+ PyErr_SetString(PyExc_OverflowError, "value too large to perform division");
+ __PYX_ERR(1, 917, __pyx_L1_error)
+ }
+ __pyx_v_shape = __Pyx_div_Py_ssize_t(__pyx_v_view->len, __pyx_v_itemsize);
+
+ /* "View.MemoryView":918
+ * if view.ndim == 0:
+ * shape = view.len / itemsize
+ * stride = itemsize # <<<<<<<<<<<<<<
+ * else:
+ * shape = view.shape[dim]
+ */
+ __pyx_v_stride = __pyx_v_itemsize;
+
+ /* "View.MemoryView":916
+ * cdef char *resultp
+ *
+ * if view.ndim == 0: # <<<<<<<<<<<<<<
+ * shape = view.len / itemsize
+ * stride = itemsize
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":920
+ * stride = itemsize
+ * else:
+ * shape = view.shape[dim] # <<<<<<<<<<<<<<
+ * stride = view.strides[dim]
+ * if view.suboffsets != NULL:
+ */
+ /*else*/ {
+ __pyx_v_shape = (__pyx_v_view->shape[__pyx_v_dim]);
+
+ /* "View.MemoryView":921
+ * else:
+ * shape = view.shape[dim]
+ * stride = view.strides[dim] # <<<<<<<<<<<<<<
+ * if view.suboffsets != NULL:
+ * suboffset = view.suboffsets[dim]
+ */
+ __pyx_v_stride = (__pyx_v_view->strides[__pyx_v_dim]);
+
+ /* "View.MemoryView":922
+ * shape = view.shape[dim]
+ * stride = view.strides[dim]
+ * if view.suboffsets != NULL: # <<<<<<<<<<<<<<
+ * suboffset = view.suboffsets[dim]
+ *
+ */
+ __pyx_t_2 = ((__pyx_v_view->suboffsets != NULL) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":923
+ * stride = view.strides[dim]
+ * if view.suboffsets != NULL:
+ * suboffset = view.suboffsets[dim] # <<<<<<<<<<<<<<
+ *
+ * if index < 0:
+ */
+ __pyx_v_suboffset = (__pyx_v_view->suboffsets[__pyx_v_dim]);
+
+ /* "View.MemoryView":922
+ * shape = view.shape[dim]
+ * stride = view.strides[dim]
+ * if view.suboffsets != NULL: # <<<<<<<<<<<<<<
+ * suboffset = view.suboffsets[dim]
+ *
+ */
+ }
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":925
+ * suboffset = view.suboffsets[dim]
+ *
+ * if index < 0: # <<<<<<<<<<<<<<
+ * index += view.shape[dim]
+ * if index < 0:
+ */
+ __pyx_t_2 = ((__pyx_v_index < 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":926
+ *
+ * if index < 0:
+ * index += view.shape[dim] # <<<<<<<<<<<<<<
+ * if index < 0:
+ * raise IndexError("Out of bounds on buffer access (axis %d)" % dim)
+ */
+ __pyx_v_index = (__pyx_v_index + (__pyx_v_view->shape[__pyx_v_dim]));
+
+ /* "View.MemoryView":927
+ * if index < 0:
+ * index += view.shape[dim]
+ * if index < 0: # <<<<<<<<<<<<<<
+ * raise IndexError("Out of bounds on buffer access (axis %d)" % dim)
+ *
+ */
+ __pyx_t_2 = ((__pyx_v_index < 0) != 0);
+ if (unlikely(__pyx_t_2)) {
+
+ /* "View.MemoryView":928
+ * index += view.shape[dim]
+ * if index < 0:
+ * raise IndexError("Out of bounds on buffer access (axis %d)" % dim) # <<<<<<<<<<<<<<
+ *
+ * if index >= shape:
+ */
+ __pyx_t_3 = PyInt_FromSsize_t(__pyx_v_dim); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 928, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_t_4 = __Pyx_PyString_Format(__pyx_kp_s_Out_of_bounds_on_buffer_access_a, __pyx_t_3); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 928, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __pyx_t_3 = __Pyx_PyObject_CallOneArg(__pyx_builtin_IndexError, __pyx_t_4); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 928, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+ __Pyx_Raise(__pyx_t_3, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __PYX_ERR(1, 928, __pyx_L1_error)
+
+ /* "View.MemoryView":927
+ * if index < 0:
+ * index += view.shape[dim]
+ * if index < 0: # <<<<<<<<<<<<<<
+ * raise IndexError("Out of bounds on buffer access (axis %d)" % dim)
+ *
+ */
+ }
+
+ /* "View.MemoryView":925
+ * suboffset = view.suboffsets[dim]
+ *
+ * if index < 0: # <<<<<<<<<<<<<<
+ * index += view.shape[dim]
+ * if index < 0:
+ */
+ }
+
+ /* "View.MemoryView":930
+ * raise IndexError("Out of bounds on buffer access (axis %d)" % dim)
+ *
+ * if index >= shape: # <<<<<<<<<<<<<<
+ * raise IndexError("Out of bounds on buffer access (axis %d)" % dim)
+ *
+ */
+ __pyx_t_2 = ((__pyx_v_index >= __pyx_v_shape) != 0);
+ if (unlikely(__pyx_t_2)) {
+
+ /* "View.MemoryView":931
+ *
+ * if index >= shape:
+ * raise IndexError("Out of bounds on buffer access (axis %d)" % dim) # <<<<<<<<<<<<<<
+ *
+ * resultp = bufp + index * stride
+ */
+ __pyx_t_3 = PyInt_FromSsize_t(__pyx_v_dim); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 931, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __pyx_t_4 = __Pyx_PyString_Format(__pyx_kp_s_Out_of_bounds_on_buffer_access_a, __pyx_t_3); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 931, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_4);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __pyx_t_3 = __Pyx_PyObject_CallOneArg(__pyx_builtin_IndexError, __pyx_t_4); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 931, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
+ __Pyx_Raise(__pyx_t_3, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __PYX_ERR(1, 931, __pyx_L1_error)
+
+ /* "View.MemoryView":930
+ * raise IndexError("Out of bounds on buffer access (axis %d)" % dim)
+ *
+ * if index >= shape: # <<<<<<<<<<<<<<
+ * raise IndexError("Out of bounds on buffer access (axis %d)" % dim)
+ *
+ */
+ }
+
+ /* "View.MemoryView":933
+ * raise IndexError("Out of bounds on buffer access (axis %d)" % dim)
+ *
+ * resultp = bufp + index * stride # <<<<<<<<<<<<<<
+ * if suboffset >= 0:
+ * resultp = ( resultp)[0] + suboffset
+ */
+ __pyx_v_resultp = (__pyx_v_bufp + (__pyx_v_index * __pyx_v_stride));
+
+ /* "View.MemoryView":934
+ *
+ * resultp = bufp + index * stride
+ * if suboffset >= 0: # <<<<<<<<<<<<<<
+ * resultp = ( resultp)[0] + suboffset
+ *
+ */
+ __pyx_t_2 = ((__pyx_v_suboffset >= 0) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":935
+ * resultp = bufp + index * stride
+ * if suboffset >= 0:
+ * resultp = ( resultp)[0] + suboffset # <<<<<<<<<<<<<<
+ *
+ * return resultp
+ */
+ __pyx_v_resultp = ((((char **)__pyx_v_resultp)[0]) + __pyx_v_suboffset);
+
+ /* "View.MemoryView":934
+ *
+ * resultp = bufp + index * stride
+ * if suboffset >= 0: # <<<<<<<<<<<<<<
+ * resultp = ( resultp)[0] + suboffset
+ *
+ */
+ }
+
+ /* "View.MemoryView":937
+ * resultp = ( resultp)[0] + suboffset
+ *
+ * return resultp # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_r = __pyx_v_resultp;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":910
+ *
+ * @cname('__pyx_pybuffer_index')
+ * cdef char *pybuffer_index(Py_buffer *view, char *bufp, Py_ssize_t index, # <<<<<<<<<<<<<<
+ * Py_ssize_t dim) except NULL:
+ * cdef Py_ssize_t shape, stride, suboffset = -1
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_XDECREF(__pyx_t_4);
+ __Pyx_AddTraceback("View.MemoryView.pybuffer_index", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":943
+ *
+ * @cname('__pyx_memslice_transpose')
+ * cdef int transpose_memslice(__Pyx_memviewslice *memslice) nogil except 0: # <<<<<<<<<<<<<<
+ * cdef int ndim = memslice.memview.view.ndim
+ *
+ */
+
+static int __pyx_memslice_transpose(__Pyx_memviewslice *__pyx_v_memslice) {
+ int __pyx_v_ndim;
+ Py_ssize_t *__pyx_v_shape;
+ Py_ssize_t *__pyx_v_strides;
+ int __pyx_v_i;
+ int __pyx_v_j;
+ int __pyx_r;
+ int __pyx_t_1;
+ Py_ssize_t *__pyx_t_2;
+ long __pyx_t_3;
+ long __pyx_t_4;
+ Py_ssize_t __pyx_t_5;
+ Py_ssize_t __pyx_t_6;
+ int __pyx_t_7;
+ int __pyx_t_8;
+ int __pyx_t_9;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+
+ /* "View.MemoryView":944
+ * @cname('__pyx_memslice_transpose')
+ * cdef int transpose_memslice(__Pyx_memviewslice *memslice) nogil except 0:
+ * cdef int ndim = memslice.memview.view.ndim # <<<<<<<<<<<<<<
+ *
+ * cdef Py_ssize_t *shape = memslice.shape
+ */
+ __pyx_t_1 = __pyx_v_memslice->memview->view.ndim;
+ __pyx_v_ndim = __pyx_t_1;
+
+ /* "View.MemoryView":946
+ * cdef int ndim = memslice.memview.view.ndim
+ *
+ * cdef Py_ssize_t *shape = memslice.shape # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t *strides = memslice.strides
+ *
+ */
+ __pyx_t_2 = __pyx_v_memslice->shape;
+ __pyx_v_shape = __pyx_t_2;
+
+ /* "View.MemoryView":947
+ *
+ * cdef Py_ssize_t *shape = memslice.shape
+ * cdef Py_ssize_t *strides = memslice.strides # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_t_2 = __pyx_v_memslice->strides;
+ __pyx_v_strides = __pyx_t_2;
+
+ /* "View.MemoryView":951
+ *
+ * cdef int i, j
+ * for i in range(ndim / 2): # <<<<<<<<<<<<<<
+ * j = ndim - 1 - i
+ * strides[i], strides[j] = strides[j], strides[i]
+ */
+ __pyx_t_3 = __Pyx_div_long(__pyx_v_ndim, 2);
+ __pyx_t_4 = __pyx_t_3;
+ for (__pyx_t_1 = 0; __pyx_t_1 < __pyx_t_4; __pyx_t_1+=1) {
+ __pyx_v_i = __pyx_t_1;
+
+ /* "View.MemoryView":952
+ * cdef int i, j
+ * for i in range(ndim / 2):
+ * j = ndim - 1 - i # <<<<<<<<<<<<<<
+ * strides[i], strides[j] = strides[j], strides[i]
+ * shape[i], shape[j] = shape[j], shape[i]
+ */
+ __pyx_v_j = ((__pyx_v_ndim - 1) - __pyx_v_i);
+
+ /* "View.MemoryView":953
+ * for i in range(ndim / 2):
+ * j = ndim - 1 - i
+ * strides[i], strides[j] = strides[j], strides[i] # <<<<<<<<<<<<<<
+ * shape[i], shape[j] = shape[j], shape[i]
+ *
+ */
+ __pyx_t_5 = (__pyx_v_strides[__pyx_v_j]);
+ __pyx_t_6 = (__pyx_v_strides[__pyx_v_i]);
+ (__pyx_v_strides[__pyx_v_i]) = __pyx_t_5;
+ (__pyx_v_strides[__pyx_v_j]) = __pyx_t_6;
+
+ /* "View.MemoryView":954
+ * j = ndim - 1 - i
+ * strides[i], strides[j] = strides[j], strides[i]
+ * shape[i], shape[j] = shape[j], shape[i] # <<<<<<<<<<<<<<
+ *
+ * if memslice.suboffsets[i] >= 0 or memslice.suboffsets[j] >= 0:
+ */
+ __pyx_t_6 = (__pyx_v_shape[__pyx_v_j]);
+ __pyx_t_5 = (__pyx_v_shape[__pyx_v_i]);
+ (__pyx_v_shape[__pyx_v_i]) = __pyx_t_6;
+ (__pyx_v_shape[__pyx_v_j]) = __pyx_t_5;
+
+ /* "View.MemoryView":956
+ * shape[i], shape[j] = shape[j], shape[i]
+ *
+ * if memslice.suboffsets[i] >= 0 or memslice.suboffsets[j] >= 0: # <<<<<<<<<<<<<<
+ * _err(ValueError, "Cannot transpose memoryview with indirect dimensions")
+ *
+ */
+ __pyx_t_8 = (((__pyx_v_memslice->suboffsets[__pyx_v_i]) >= 0) != 0);
+ if (!__pyx_t_8) {
+ } else {
+ __pyx_t_7 = __pyx_t_8;
+ goto __pyx_L6_bool_binop_done;
+ }
+ __pyx_t_8 = (((__pyx_v_memslice->suboffsets[__pyx_v_j]) >= 0) != 0);
+ __pyx_t_7 = __pyx_t_8;
+ __pyx_L6_bool_binop_done:;
+ if (__pyx_t_7) {
+
+ /* "View.MemoryView":957
+ *
+ * if memslice.suboffsets[i] >= 0 or memslice.suboffsets[j] >= 0:
+ * _err(ValueError, "Cannot transpose memoryview with indirect dimensions") # <<<<<<<<<<<<<<
+ *
+ * return 1
+ */
+ __pyx_t_9 = __pyx_memoryview_err(__pyx_builtin_ValueError, ((char *)"Cannot transpose memoryview with indirect dimensions")); if (unlikely(__pyx_t_9 == ((int)-1))) __PYX_ERR(1, 957, __pyx_L1_error)
+
+ /* "View.MemoryView":956
+ * shape[i], shape[j] = shape[j], shape[i]
+ *
+ * if memslice.suboffsets[i] >= 0 or memslice.suboffsets[j] >= 0: # <<<<<<<<<<<<<<
+ * _err(ValueError, "Cannot transpose memoryview with indirect dimensions")
+ *
+ */
+ }
+ }
+
+ /* "View.MemoryView":959
+ * _err(ValueError, "Cannot transpose memoryview with indirect dimensions")
+ *
+ * return 1 # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_r = 1;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":943
+ *
+ * @cname('__pyx_memslice_transpose')
+ * cdef int transpose_memslice(__Pyx_memviewslice *memslice) nogil except 0: # <<<<<<<<<<<<<<
+ * cdef int ndim = memslice.memview.view.ndim
+ *
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ {
+ #ifdef WITH_THREAD
+ PyGILState_STATE __pyx_gilstate_save = __Pyx_PyGILState_Ensure();
+ #endif
+ __Pyx_AddTraceback("View.MemoryView.transpose_memslice", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ #ifdef WITH_THREAD
+ __Pyx_PyGILState_Release(__pyx_gilstate_save);
+ #endif
+ }
+ __pyx_r = 0;
+ __pyx_L0:;
+ return __pyx_r;
+}
+
+/* "View.MemoryView":976
+ * cdef int (*to_dtype_func)(char *, object) except 0
+ *
+ * def __dealloc__(self): # <<<<<<<<<<<<<<
+ * __PYX_XDEC_MEMVIEW(&self.from_slice, 1)
+ *
+ */
+
+/* Python wrapper */
+static void __pyx_memoryviewslice___dealloc__(PyObject *__pyx_v_self); /*proto*/
+static void __pyx_memoryviewslice___dealloc__(PyObject *__pyx_v_self) {
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__dealloc__ (wrapper)", 0);
+ __pyx_memoryviewslice___pyx_pf_15View_dot_MemoryView_16_memoryviewslice___dealloc__(((struct __pyx_memoryviewslice_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+}
+
+static void __pyx_memoryviewslice___pyx_pf_15View_dot_MemoryView_16_memoryviewslice___dealloc__(struct __pyx_memoryviewslice_obj *__pyx_v_self) {
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__dealloc__", 0);
+
+ /* "View.MemoryView":977
+ *
+ * def __dealloc__(self):
+ * __PYX_XDEC_MEMVIEW(&self.from_slice, 1) # <<<<<<<<<<<<<<
+ *
+ * cdef convert_item_to_object(self, char *itemp):
+ */
+ __PYX_XDEC_MEMVIEW((&__pyx_v_self->from_slice), 1);
+
+ /* "View.MemoryView":976
+ * cdef int (*to_dtype_func)(char *, object) except 0
+ *
+ * def __dealloc__(self): # <<<<<<<<<<<<<<
+ * __PYX_XDEC_MEMVIEW(&self.from_slice, 1)
+ *
+ */
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+}
+
+/* "View.MemoryView":979
+ * __PYX_XDEC_MEMVIEW(&self.from_slice, 1)
+ *
+ * cdef convert_item_to_object(self, char *itemp): # <<<<<<<<<<<<<<
+ * if self.to_object_func != NULL:
+ * return self.to_object_func(itemp)
+ */
+
+static PyObject *__pyx_memoryviewslice_convert_item_to_object(struct __pyx_memoryviewslice_obj *__pyx_v_self, char *__pyx_v_itemp) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("convert_item_to_object", 0);
+
+ /* "View.MemoryView":980
+ *
+ * cdef convert_item_to_object(self, char *itemp):
+ * if self.to_object_func != NULL: # <<<<<<<<<<<<<<
+ * return self.to_object_func(itemp)
+ * else:
+ */
+ __pyx_t_1 = ((__pyx_v_self->to_object_func != NULL) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":981
+ * cdef convert_item_to_object(self, char *itemp):
+ * if self.to_object_func != NULL:
+ * return self.to_object_func(itemp) # <<<<<<<<<<<<<<
+ * else:
+ * return memoryview.convert_item_to_object(self, itemp)
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_2 = __pyx_v_self->to_object_func(__pyx_v_itemp); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 981, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":980
+ *
+ * cdef convert_item_to_object(self, char *itemp):
+ * if self.to_object_func != NULL: # <<<<<<<<<<<<<<
+ * return self.to_object_func(itemp)
+ * else:
+ */
+ }
+
+ /* "View.MemoryView":983
+ * return self.to_object_func(itemp)
+ * else:
+ * return memoryview.convert_item_to_object(self, itemp) # <<<<<<<<<<<<<<
+ *
+ * cdef assign_item_from_object(self, char *itemp, object value):
+ */
+ /*else*/ {
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_2 = __pyx_memoryview_convert_item_to_object(((struct __pyx_memoryview_obj *)__pyx_v_self), __pyx_v_itemp); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 983, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_r = __pyx_t_2;
+ __pyx_t_2 = 0;
+ goto __pyx_L0;
+ }
+
+ /* "View.MemoryView":979
+ * __PYX_XDEC_MEMVIEW(&self.from_slice, 1)
+ *
+ * cdef convert_item_to_object(self, char *itemp): # <<<<<<<<<<<<<<
+ * if self.to_object_func != NULL:
+ * return self.to_object_func(itemp)
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_AddTraceback("View.MemoryView._memoryviewslice.convert_item_to_object", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":985
+ * return memoryview.convert_item_to_object(self, itemp)
+ *
+ * cdef assign_item_from_object(self, char *itemp, object value): # <<<<<<<<<<<<<<
+ * if self.to_dtype_func != NULL:
+ * self.to_dtype_func(itemp, value)
+ */
+
+static PyObject *__pyx_memoryviewslice_assign_item_from_object(struct __pyx_memoryviewslice_obj *__pyx_v_self, char *__pyx_v_itemp, PyObject *__pyx_v_value) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("assign_item_from_object", 0);
+
+ /* "View.MemoryView":986
+ *
+ * cdef assign_item_from_object(self, char *itemp, object value):
+ * if self.to_dtype_func != NULL: # <<<<<<<<<<<<<<
+ * self.to_dtype_func(itemp, value)
+ * else:
+ */
+ __pyx_t_1 = ((__pyx_v_self->to_dtype_func != NULL) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":987
+ * cdef assign_item_from_object(self, char *itemp, object value):
+ * if self.to_dtype_func != NULL:
+ * self.to_dtype_func(itemp, value) # <<<<<<<<<<<<<<
+ * else:
+ * memoryview.assign_item_from_object(self, itemp, value)
+ */
+ __pyx_t_2 = __pyx_v_self->to_dtype_func(__pyx_v_itemp, __pyx_v_value); if (unlikely(__pyx_t_2 == ((int)0))) __PYX_ERR(1, 987, __pyx_L1_error)
+
+ /* "View.MemoryView":986
+ *
+ * cdef assign_item_from_object(self, char *itemp, object value):
+ * if self.to_dtype_func != NULL: # <<<<<<<<<<<<<<
+ * self.to_dtype_func(itemp, value)
+ * else:
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":989
+ * self.to_dtype_func(itemp, value)
+ * else:
+ * memoryview.assign_item_from_object(self, itemp, value) # <<<<<<<<<<<<<<
+ *
+ * @property
+ */
+ /*else*/ {
+ __pyx_t_3 = __pyx_memoryview_assign_item_from_object(((struct __pyx_memoryview_obj *)__pyx_v_self), __pyx_v_itemp, __pyx_v_value); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 989, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":985
+ * return memoryview.convert_item_to_object(self, itemp)
+ *
+ * cdef assign_item_from_object(self, char *itemp, object value): # <<<<<<<<<<<<<<
+ * if self.to_dtype_func != NULL:
+ * self.to_dtype_func(itemp, value)
+ */
+
+ /* function exit code */
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView._memoryviewslice.assign_item_from_object", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":992
+ *
+ * @property
+ * def base(self): # <<<<<<<<<<<<<<
+ * return self.from_object
+ *
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw_15View_dot_MemoryView_16_memoryviewslice_4base_1__get__(PyObject *__pyx_v_self); /*proto*/
+static PyObject *__pyx_pw_15View_dot_MemoryView_16_memoryviewslice_4base_1__get__(PyObject *__pyx_v_self) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__ (wrapper)", 0);
+ __pyx_r = __pyx_pf_15View_dot_MemoryView_16_memoryviewslice_4base___get__(((struct __pyx_memoryviewslice_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf_15View_dot_MemoryView_16_memoryviewslice_4base___get__(struct __pyx_memoryviewslice_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__get__", 0);
+
+ /* "View.MemoryView":993
+ * @property
+ * def base(self):
+ * return self.from_object # <<<<<<<<<<<<<<
+ *
+ * __pyx_getbuffer = capsule( &__pyx_memoryview_getbuffer, "getbuffer(obj, view, flags)")
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(__pyx_v_self->from_object);
+ __pyx_r = __pyx_v_self->from_object;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":992
+ *
+ * @property
+ * def base(self): # <<<<<<<<<<<<<<
+ * return self.from_object
+ *
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "(tree fragment)":1
+ * def __reduce_cython__(self): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw___pyx_memoryviewslice_1__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/
+static PyObject *__pyx_pw___pyx_memoryviewslice_1__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0);
+ __pyx_r = __pyx_pf___pyx_memoryviewslice___reduce_cython__(((struct __pyx_memoryviewslice_obj *)__pyx_v_self));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf___pyx_memoryviewslice___reduce_cython__(CYTHON_UNUSED struct __pyx_memoryviewslice_obj *__pyx_v_self) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__reduce_cython__", 0);
+
+ /* "(tree fragment)":2
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<<
+ * def __setstate_cython__(self, __pyx_state):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__21, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 2, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_Raise(__pyx_t_1, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __PYX_ERR(1, 2, __pyx_L1_error)
+
+ /* "(tree fragment)":1
+ * def __reduce_cython__(self): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView._memoryviewslice.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "(tree fragment)":3
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+
+/* Python wrapper */
+static PyObject *__pyx_pw___pyx_memoryviewslice_3__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/
+static PyObject *__pyx_pw___pyx_memoryviewslice_3__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) {
+ PyObject *__pyx_r = 0;
+ __Pyx_RefNannyDeclarations
+ __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0);
+ __pyx_r = __pyx_pf___pyx_memoryviewslice_2__setstate_cython__(((struct __pyx_memoryviewslice_obj *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state));
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+static PyObject *__pyx_pf___pyx_memoryviewslice_2__setstate_cython__(CYTHON_UNUSED struct __pyx_memoryviewslice_obj *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) {
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("__setstate_cython__", 0);
+
+ /* "(tree fragment)":4
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<<
+ */
+ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__22, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __Pyx_Raise(__pyx_t_1, 0, 0, 0);
+ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+ __PYX_ERR(1, 4, __pyx_L1_error)
+
+ /* "(tree fragment)":3
+ * def __reduce_cython__(self):
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<<
+ * raise TypeError("no default __reduce__ due to non-trivial __cinit__")
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView._memoryviewslice.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":999
+ *
+ * @cname('__pyx_memoryview_fromslice')
+ * cdef memoryview_fromslice(__Pyx_memviewslice memviewslice, # <<<<<<<<<<<<<<
+ * int ndim,
+ * object (*to_object_func)(char *),
+ */
+
+static PyObject *__pyx_memoryview_fromslice(__Pyx_memviewslice __pyx_v_memviewslice, int __pyx_v_ndim, PyObject *(*__pyx_v_to_object_func)(char *), int (*__pyx_v_to_dtype_func)(char *, PyObject *), int __pyx_v_dtype_is_object) {
+ struct __pyx_memoryviewslice_obj *__pyx_v_result = 0;
+ Py_ssize_t __pyx_v_suboffset;
+ PyObject *__pyx_v_length = NULL;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ PyObject *__pyx_t_2 = NULL;
+ PyObject *__pyx_t_3 = NULL;
+ __Pyx_TypeInfo *__pyx_t_4;
+ Py_buffer __pyx_t_5;
+ Py_ssize_t *__pyx_t_6;
+ Py_ssize_t *__pyx_t_7;
+ Py_ssize_t *__pyx_t_8;
+ Py_ssize_t __pyx_t_9;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("memoryview_fromslice", 0);
+
+ /* "View.MemoryView":1007
+ * cdef _memoryviewslice result
+ *
+ * if memviewslice.memview == Py_None: # <<<<<<<<<<<<<<
+ * return None
+ *
+ */
+ __pyx_t_1 = ((((PyObject *)__pyx_v_memviewslice.memview) == Py_None) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":1008
+ *
+ * if memviewslice.memview == Py_None:
+ * return None # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_r = Py_None; __Pyx_INCREF(Py_None);
+ goto __pyx_L0;
+
+ /* "View.MemoryView":1007
+ * cdef _memoryviewslice result
+ *
+ * if memviewslice.memview == Py_None: # <<<<<<<<<<<<<<
+ * return None
+ *
+ */
+ }
+
+ /* "View.MemoryView":1013
+ *
+ *
+ * result = _memoryviewslice(None, 0, dtype_is_object) # <<<<<<<<<<<<<<
+ *
+ * result.from_slice = memviewslice
+ */
+ __pyx_t_2 = __Pyx_PyBool_FromLong(__pyx_v_dtype_is_object); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 1013, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_3 = PyTuple_New(3); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 1013, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_INCREF(Py_None);
+ __Pyx_GIVEREF(Py_None);
+ PyTuple_SET_ITEM(__pyx_t_3, 0, Py_None);
+ __Pyx_INCREF(__pyx_int_0);
+ __Pyx_GIVEREF(__pyx_int_0);
+ PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_int_0);
+ __Pyx_GIVEREF(__pyx_t_2);
+ PyTuple_SET_ITEM(__pyx_t_3, 2, __pyx_t_2);
+ __pyx_t_2 = 0;
+ __pyx_t_2 = __Pyx_PyObject_Call(((PyObject *)__pyx_memoryviewslice_type), __pyx_t_3, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 1013, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __pyx_v_result = ((struct __pyx_memoryviewslice_obj *)__pyx_t_2);
+ __pyx_t_2 = 0;
+
+ /* "View.MemoryView":1015
+ * result = _memoryviewslice(None, 0, dtype_is_object)
+ *
+ * result.from_slice = memviewslice # <<<<<<<<<<<<<<
+ * __PYX_INC_MEMVIEW(&memviewslice, 1)
+ *
+ */
+ __pyx_v_result->from_slice = __pyx_v_memviewslice;
+
+ /* "View.MemoryView":1016
+ *
+ * result.from_slice = memviewslice
+ * __PYX_INC_MEMVIEW(&memviewslice, 1) # <<<<<<<<<<<<<<
+ *
+ * result.from_object = ( memviewslice.memview).base
+ */
+ __PYX_INC_MEMVIEW((&__pyx_v_memviewslice), 1);
+
+ /* "View.MemoryView":1018
+ * __PYX_INC_MEMVIEW(&memviewslice, 1)
+ *
+ * result.from_object = ( memviewslice.memview).base # <<<<<<<<<<<<<<
+ * result.typeinfo = memviewslice.memview.typeinfo
+ *
+ */
+ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_memviewslice.memview), __pyx_n_s_base); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 1018, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_GIVEREF(__pyx_t_2);
+ __Pyx_GOTREF(__pyx_v_result->from_object);
+ __Pyx_DECREF(__pyx_v_result->from_object);
+ __pyx_v_result->from_object = __pyx_t_2;
+ __pyx_t_2 = 0;
+
+ /* "View.MemoryView":1019
+ *
+ * result.from_object = ( memviewslice.memview).base
+ * result.typeinfo = memviewslice.memview.typeinfo # <<<<<<<<<<<<<<
+ *
+ * result.view = memviewslice.memview.view
+ */
+ __pyx_t_4 = __pyx_v_memviewslice.memview->typeinfo;
+ __pyx_v_result->__pyx_base.typeinfo = __pyx_t_4;
+
+ /* "View.MemoryView":1021
+ * result.typeinfo = memviewslice.memview.typeinfo
+ *
+ * result.view = memviewslice.memview.view # <<<<<<<<<<<<<<
+ * result.view.buf = memviewslice.data
+ * result.view.ndim = ndim
+ */
+ __pyx_t_5 = __pyx_v_memviewslice.memview->view;
+ __pyx_v_result->__pyx_base.view = __pyx_t_5;
+
+ /* "View.MemoryView":1022
+ *
+ * result.view = memviewslice.memview.view
+ * result.view.buf = memviewslice.data # <<<<<<<<<<<<<<
+ * result.view.ndim = ndim
+ * (<__pyx_buffer *> &result.view).obj = Py_None
+ */
+ __pyx_v_result->__pyx_base.view.buf = ((void *)__pyx_v_memviewslice.data);
+
+ /* "View.MemoryView":1023
+ * result.view = memviewslice.memview.view
+ * result.view.buf = memviewslice.data
+ * result.view.ndim = ndim # <<<<<<<<<<<<<<
+ * (<__pyx_buffer *> &result.view).obj = Py_None
+ * Py_INCREF(Py_None)
+ */
+ __pyx_v_result->__pyx_base.view.ndim = __pyx_v_ndim;
+
+ /* "View.MemoryView":1024
+ * result.view.buf = memviewslice.data
+ * result.view.ndim = ndim
+ * (<__pyx_buffer *> &result.view).obj = Py_None # <<<<<<<<<<<<<<
+ * Py_INCREF(Py_None)
+ *
+ */
+ ((Py_buffer *)(&__pyx_v_result->__pyx_base.view))->obj = Py_None;
+
+ /* "View.MemoryView":1025
+ * result.view.ndim = ndim
+ * (<__pyx_buffer *> &result.view).obj = Py_None
+ * Py_INCREF(Py_None) # <<<<<<<<<<<<<<
+ *
+ * if (memviewslice.memview).flags & PyBUF_WRITABLE:
+ */
+ Py_INCREF(Py_None);
+
+ /* "View.MemoryView":1027
+ * Py_INCREF(Py_None)
+ *
+ * if (memviewslice.memview).flags & PyBUF_WRITABLE: # <<<<<<<<<<<<<<
+ * result.flags = PyBUF_RECORDS
+ * else:
+ */
+ __pyx_t_1 = ((((struct __pyx_memoryview_obj *)__pyx_v_memviewslice.memview)->flags & PyBUF_WRITABLE) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":1028
+ *
+ * if (memviewslice.memview).flags & PyBUF_WRITABLE:
+ * result.flags = PyBUF_RECORDS # <<<<<<<<<<<<<<
+ * else:
+ * result.flags = PyBUF_RECORDS_RO
+ */
+ __pyx_v_result->__pyx_base.flags = PyBUF_RECORDS;
+
+ /* "View.MemoryView":1027
+ * Py_INCREF(Py_None)
+ *
+ * if (memviewslice.memview).flags & PyBUF_WRITABLE: # <<<<<<<<<<<<<<
+ * result.flags = PyBUF_RECORDS
+ * else:
+ */
+ goto __pyx_L4;
+ }
+
+ /* "View.MemoryView":1030
+ * result.flags = PyBUF_RECORDS
+ * else:
+ * result.flags = PyBUF_RECORDS_RO # <<<<<<<<<<<<<<
+ *
+ * result.view.shape = result.from_slice.shape
+ */
+ /*else*/ {
+ __pyx_v_result->__pyx_base.flags = PyBUF_RECORDS_RO;
+ }
+ __pyx_L4:;
+
+ /* "View.MemoryView":1032
+ * result.flags = PyBUF_RECORDS_RO
+ *
+ * result.view.shape = result.from_slice.shape # <<<<<<<<<<<<<<
+ * result.view.strides = result.from_slice.strides
+ *
+ */
+ __pyx_v_result->__pyx_base.view.shape = ((Py_ssize_t *)__pyx_v_result->from_slice.shape);
+
+ /* "View.MemoryView":1033
+ *
+ * result.view.shape = result.from_slice.shape
+ * result.view.strides = result.from_slice.strides # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_v_result->__pyx_base.view.strides = ((Py_ssize_t *)__pyx_v_result->from_slice.strides);
+
+ /* "View.MemoryView":1036
+ *
+ *
+ * result.view.suboffsets = NULL # <<<<<<<<<<<<<<
+ * for suboffset in result.from_slice.suboffsets[:ndim]:
+ * if suboffset >= 0:
+ */
+ __pyx_v_result->__pyx_base.view.suboffsets = NULL;
+
+ /* "View.MemoryView":1037
+ *
+ * result.view.suboffsets = NULL
+ * for suboffset in result.from_slice.suboffsets[:ndim]: # <<<<<<<<<<<<<<
+ * if suboffset >= 0:
+ * result.view.suboffsets = result.from_slice.suboffsets
+ */
+ __pyx_t_7 = (__pyx_v_result->from_slice.suboffsets + __pyx_v_ndim);
+ for (__pyx_t_8 = __pyx_v_result->from_slice.suboffsets; __pyx_t_8 < __pyx_t_7; __pyx_t_8++) {
+ __pyx_t_6 = __pyx_t_8;
+ __pyx_v_suboffset = (__pyx_t_6[0]);
+
+ /* "View.MemoryView":1038
+ * result.view.suboffsets = NULL
+ * for suboffset in result.from_slice.suboffsets[:ndim]:
+ * if suboffset >= 0: # <<<<<<<<<<<<<<
+ * result.view.suboffsets = result.from_slice.suboffsets
+ * break
+ */
+ __pyx_t_1 = ((__pyx_v_suboffset >= 0) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":1039
+ * for suboffset in result.from_slice.suboffsets[:ndim]:
+ * if suboffset >= 0:
+ * result.view.suboffsets = result.from_slice.suboffsets # <<<<<<<<<<<<<<
+ * break
+ *
+ */
+ __pyx_v_result->__pyx_base.view.suboffsets = ((Py_ssize_t *)__pyx_v_result->from_slice.suboffsets);
+
+ /* "View.MemoryView":1040
+ * if suboffset >= 0:
+ * result.view.suboffsets = result.from_slice.suboffsets
+ * break # <<<<<<<<<<<<<<
+ *
+ * result.view.len = result.view.itemsize
+ */
+ goto __pyx_L6_break;
+
+ /* "View.MemoryView":1038
+ * result.view.suboffsets = NULL
+ * for suboffset in result.from_slice.suboffsets[:ndim]:
+ * if suboffset >= 0: # <<<<<<<<<<<<<<
+ * result.view.suboffsets = result.from_slice.suboffsets
+ * break
+ */
+ }
+ }
+ __pyx_L6_break:;
+
+ /* "View.MemoryView":1042
+ * break
+ *
+ * result.view.len = result.view.itemsize # <<<<<<<<<<<<<<
+ * for length in result.view.shape[:ndim]:
+ * result.view.len *= length
+ */
+ __pyx_t_9 = __pyx_v_result->__pyx_base.view.itemsize;
+ __pyx_v_result->__pyx_base.view.len = __pyx_t_9;
+
+ /* "View.MemoryView":1043
+ *
+ * result.view.len = result.view.itemsize
+ * for length in result.view.shape[:ndim]: # <<<<<<<<<<<<<<
+ * result.view.len *= length
+ *
+ */
+ __pyx_t_7 = (__pyx_v_result->__pyx_base.view.shape + __pyx_v_ndim);
+ for (__pyx_t_8 = __pyx_v_result->__pyx_base.view.shape; __pyx_t_8 < __pyx_t_7; __pyx_t_8++) {
+ __pyx_t_6 = __pyx_t_8;
+ __pyx_t_2 = PyInt_FromSsize_t((__pyx_t_6[0])); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 1043, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __Pyx_XDECREF_SET(__pyx_v_length, __pyx_t_2);
+ __pyx_t_2 = 0;
+
+ /* "View.MemoryView":1044
+ * result.view.len = result.view.itemsize
+ * for length in result.view.shape[:ndim]:
+ * result.view.len *= length # <<<<<<<<<<<<<<
+ *
+ * result.to_object_func = to_object_func
+ */
+ __pyx_t_2 = PyInt_FromSsize_t(__pyx_v_result->__pyx_base.view.len); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 1044, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_2);
+ __pyx_t_3 = PyNumber_InPlaceMultiply(__pyx_t_2, __pyx_v_length); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 1044, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_3);
+ __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+ __pyx_t_9 = __Pyx_PyIndex_AsSsize_t(__pyx_t_3); if (unlikely((__pyx_t_9 == (Py_ssize_t)-1) && PyErr_Occurred())) __PYX_ERR(1, 1044, __pyx_L1_error)
+ __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
+ __pyx_v_result->__pyx_base.view.len = __pyx_t_9;
+ }
+
+ /* "View.MemoryView":1046
+ * result.view.len *= length
+ *
+ * result.to_object_func = to_object_func # <<<<<<<<<<<<<<
+ * result.to_dtype_func = to_dtype_func
+ *
+ */
+ __pyx_v_result->to_object_func = __pyx_v_to_object_func;
+
+ /* "View.MemoryView":1047
+ *
+ * result.to_object_func = to_object_func
+ * result.to_dtype_func = to_dtype_func # <<<<<<<<<<<<<<
+ *
+ * return result
+ */
+ __pyx_v_result->to_dtype_func = __pyx_v_to_dtype_func;
+
+ /* "View.MemoryView":1049
+ * result.to_dtype_func = to_dtype_func
+ *
+ * return result # <<<<<<<<<<<<<<
+ *
+ * @cname('__pyx_memoryview_get_slice_from_memoryview')
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __Pyx_INCREF(((PyObject *)__pyx_v_result));
+ __pyx_r = ((PyObject *)__pyx_v_result);
+ goto __pyx_L0;
+
+ /* "View.MemoryView":999
+ *
+ * @cname('__pyx_memoryview_fromslice')
+ * cdef memoryview_fromslice(__Pyx_memviewslice memviewslice, # <<<<<<<<<<<<<<
+ * int ndim,
+ * object (*to_object_func)(char *),
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_2);
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.memoryview_fromslice", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XDECREF((PyObject *)__pyx_v_result);
+ __Pyx_XDECREF(__pyx_v_length);
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":1052
+ *
+ * @cname('__pyx_memoryview_get_slice_from_memoryview')
+ * cdef __Pyx_memviewslice *get_slice_from_memview(memoryview memview, # <<<<<<<<<<<<<<
+ * __Pyx_memviewslice *mslice) except NULL:
+ * cdef _memoryviewslice obj
+ */
+
+static __Pyx_memviewslice *__pyx_memoryview_get_slice_from_memoryview(struct __pyx_memoryview_obj *__pyx_v_memview, __Pyx_memviewslice *__pyx_v_mslice) {
+ struct __pyx_memoryviewslice_obj *__pyx_v_obj = 0;
+ __Pyx_memviewslice *__pyx_r;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *__pyx_t_3 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("get_slice_from_memview", 0);
+
+ /* "View.MemoryView":1055
+ * __Pyx_memviewslice *mslice) except NULL:
+ * cdef _memoryviewslice obj
+ * if isinstance(memview, _memoryviewslice): # <<<<<<<<<<<<<<
+ * obj = memview
+ * return &obj.from_slice
+ */
+ __pyx_t_1 = __Pyx_TypeCheck(((PyObject *)__pyx_v_memview), __pyx_memoryviewslice_type);
+ __pyx_t_2 = (__pyx_t_1 != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":1056
+ * cdef _memoryviewslice obj
+ * if isinstance(memview, _memoryviewslice):
+ * obj = memview # <<<<<<<<<<<<<<
+ * return &obj.from_slice
+ * else:
+ */
+ if (!(likely(((((PyObject *)__pyx_v_memview)) == Py_None) || likely(__Pyx_TypeTest(((PyObject *)__pyx_v_memview), __pyx_memoryviewslice_type))))) __PYX_ERR(1, 1056, __pyx_L1_error)
+ __pyx_t_3 = ((PyObject *)__pyx_v_memview);
+ __Pyx_INCREF(__pyx_t_3);
+ __pyx_v_obj = ((struct __pyx_memoryviewslice_obj *)__pyx_t_3);
+ __pyx_t_3 = 0;
+
+ /* "View.MemoryView":1057
+ * if isinstance(memview, _memoryviewslice):
+ * obj = memview
+ * return &obj.from_slice # <<<<<<<<<<<<<<
+ * else:
+ * slice_copy(memview, mslice)
+ */
+ __pyx_r = (&__pyx_v_obj->from_slice);
+ goto __pyx_L0;
+
+ /* "View.MemoryView":1055
+ * __Pyx_memviewslice *mslice) except NULL:
+ * cdef _memoryviewslice obj
+ * if isinstance(memview, _memoryviewslice): # <<<<<<<<<<<<<<
+ * obj = memview
+ * return &obj.from_slice
+ */
+ }
+
+ /* "View.MemoryView":1059
+ * return &obj.from_slice
+ * else:
+ * slice_copy(memview, mslice) # <<<<<<<<<<<<<<
+ * return mslice
+ *
+ */
+ /*else*/ {
+ __pyx_memoryview_slice_copy(__pyx_v_memview, __pyx_v_mslice);
+
+ /* "View.MemoryView":1060
+ * else:
+ * slice_copy(memview, mslice)
+ * return mslice # <<<<<<<<<<<<<<
+ *
+ * @cname('__pyx_memoryview_slice_copy')
+ */
+ __pyx_r = __pyx_v_mslice;
+ goto __pyx_L0;
+ }
+
+ /* "View.MemoryView":1052
+ *
+ * @cname('__pyx_memoryview_get_slice_from_memoryview')
+ * cdef __Pyx_memviewslice *get_slice_from_memview(memoryview memview, # <<<<<<<<<<<<<<
+ * __Pyx_memviewslice *mslice) except NULL:
+ * cdef _memoryviewslice obj
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_3);
+ __Pyx_AddTraceback("View.MemoryView.get_slice_from_memview", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = NULL;
+ __pyx_L0:;
+ __Pyx_XDECREF((PyObject *)__pyx_v_obj);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":1063
+ *
+ * @cname('__pyx_memoryview_slice_copy')
+ * cdef void slice_copy(memoryview memview, __Pyx_memviewslice *dst): # <<<<<<<<<<<<<<
+ * cdef int dim
+ * cdef (Py_ssize_t*) shape, strides, suboffsets
+ */
+
+static void __pyx_memoryview_slice_copy(struct __pyx_memoryview_obj *__pyx_v_memview, __Pyx_memviewslice *__pyx_v_dst) {
+ int __pyx_v_dim;
+ Py_ssize_t *__pyx_v_shape;
+ Py_ssize_t *__pyx_v_strides;
+ Py_ssize_t *__pyx_v_suboffsets;
+ __Pyx_RefNannyDeclarations
+ Py_ssize_t *__pyx_t_1;
+ int __pyx_t_2;
+ int __pyx_t_3;
+ int __pyx_t_4;
+ Py_ssize_t __pyx_t_5;
+ __Pyx_RefNannySetupContext("slice_copy", 0);
+
+ /* "View.MemoryView":1067
+ * cdef (Py_ssize_t*) shape, strides, suboffsets
+ *
+ * shape = memview.view.shape # <<<<<<<<<<<<<<
+ * strides = memview.view.strides
+ * suboffsets = memview.view.suboffsets
+ */
+ __pyx_t_1 = __pyx_v_memview->view.shape;
+ __pyx_v_shape = __pyx_t_1;
+
+ /* "View.MemoryView":1068
+ *
+ * shape = memview.view.shape
+ * strides = memview.view.strides # <<<<<<<<<<<<<<
+ * suboffsets = memview.view.suboffsets
+ *
+ */
+ __pyx_t_1 = __pyx_v_memview->view.strides;
+ __pyx_v_strides = __pyx_t_1;
+
+ /* "View.MemoryView":1069
+ * shape = memview.view.shape
+ * strides = memview.view.strides
+ * suboffsets = memview.view.suboffsets # <<<<<<<<<<<<<<
+ *
+ * dst.memview = <__pyx_memoryview *> memview
+ */
+ __pyx_t_1 = __pyx_v_memview->view.suboffsets;
+ __pyx_v_suboffsets = __pyx_t_1;
+
+ /* "View.MemoryView":1071
+ * suboffsets = memview.view.suboffsets
+ *
+ * dst.memview = <__pyx_memoryview *> memview # <<<<<<<<<<<<<<
+ * dst.data = memview.view.buf
+ *
+ */
+ __pyx_v_dst->memview = ((struct __pyx_memoryview_obj *)__pyx_v_memview);
+
+ /* "View.MemoryView":1072
+ *
+ * dst.memview = <__pyx_memoryview *> memview
+ * dst.data = memview.view.buf # <<<<<<<<<<<<<<
+ *
+ * for dim in range(memview.view.ndim):
+ */
+ __pyx_v_dst->data = ((char *)__pyx_v_memview->view.buf);
+
+ /* "View.MemoryView":1074
+ * dst.data = memview.view.buf
+ *
+ * for dim in range(memview.view.ndim): # <<<<<<<<<<<<<<
+ * dst.shape[dim] = shape[dim]
+ * dst.strides[dim] = strides[dim]
+ */
+ __pyx_t_2 = __pyx_v_memview->view.ndim;
+ __pyx_t_3 = __pyx_t_2;
+ for (__pyx_t_4 = 0; __pyx_t_4 < __pyx_t_3; __pyx_t_4+=1) {
+ __pyx_v_dim = __pyx_t_4;
+
+ /* "View.MemoryView":1075
+ *
+ * for dim in range(memview.view.ndim):
+ * dst.shape[dim] = shape[dim] # <<<<<<<<<<<<<<
+ * dst.strides[dim] = strides[dim]
+ * dst.suboffsets[dim] = suboffsets[dim] if suboffsets else -1
+ */
+ (__pyx_v_dst->shape[__pyx_v_dim]) = (__pyx_v_shape[__pyx_v_dim]);
+
+ /* "View.MemoryView":1076
+ * for dim in range(memview.view.ndim):
+ * dst.shape[dim] = shape[dim]
+ * dst.strides[dim] = strides[dim] # <<<<<<<<<<<<<<
+ * dst.suboffsets[dim] = suboffsets[dim] if suboffsets else -1
+ *
+ */
+ (__pyx_v_dst->strides[__pyx_v_dim]) = (__pyx_v_strides[__pyx_v_dim]);
+
+ /* "View.MemoryView":1077
+ * dst.shape[dim] = shape[dim]
+ * dst.strides[dim] = strides[dim]
+ * dst.suboffsets[dim] = suboffsets[dim] if suboffsets else -1 # <<<<<<<<<<<<<<
+ *
+ * @cname('__pyx_memoryview_copy_object')
+ */
+ if ((__pyx_v_suboffsets != 0)) {
+ __pyx_t_5 = (__pyx_v_suboffsets[__pyx_v_dim]);
+ } else {
+ __pyx_t_5 = -1L;
+ }
+ (__pyx_v_dst->suboffsets[__pyx_v_dim]) = __pyx_t_5;
+ }
+
+ /* "View.MemoryView":1063
+ *
+ * @cname('__pyx_memoryview_slice_copy')
+ * cdef void slice_copy(memoryview memview, __Pyx_memviewslice *dst): # <<<<<<<<<<<<<<
+ * cdef int dim
+ * cdef (Py_ssize_t*) shape, strides, suboffsets
+ */
+
+ /* function exit code */
+ __Pyx_RefNannyFinishContext();
+}
+
+/* "View.MemoryView":1080
+ *
+ * @cname('__pyx_memoryview_copy_object')
+ * cdef memoryview_copy(memoryview memview): # <<<<<<<<<<<<<<
+ * "Create a new memoryview object"
+ * cdef __Pyx_memviewslice memviewslice
+ */
+
+static PyObject *__pyx_memoryview_copy_object(struct __pyx_memoryview_obj *__pyx_v_memview) {
+ __Pyx_memviewslice __pyx_v_memviewslice;
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ PyObject *__pyx_t_1 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("memoryview_copy", 0);
+
+ /* "View.MemoryView":1083
+ * "Create a new memoryview object"
+ * cdef __Pyx_memviewslice memviewslice
+ * slice_copy(memview, &memviewslice) # <<<<<<<<<<<<<<
+ * return memoryview_copy_from_slice(memview, &memviewslice)
+ *
+ */
+ __pyx_memoryview_slice_copy(__pyx_v_memview, (&__pyx_v_memviewslice));
+
+ /* "View.MemoryView":1084
+ * cdef __Pyx_memviewslice memviewslice
+ * slice_copy(memview, &memviewslice)
+ * return memoryview_copy_from_slice(memview, &memviewslice) # <<<<<<<<<<<<<<
+ *
+ * @cname('__pyx_memoryview_copy_object_from_slice')
+ */
+ __Pyx_XDECREF(__pyx_r);
+ __pyx_t_1 = __pyx_memoryview_copy_object_from_slice(__pyx_v_memview, (&__pyx_v_memviewslice)); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 1084, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_1);
+ __pyx_r = __pyx_t_1;
+ __pyx_t_1 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":1080
+ *
+ * @cname('__pyx_memoryview_copy_object')
+ * cdef memoryview_copy(memoryview memview): # <<<<<<<<<<<<<<
+ * "Create a new memoryview object"
+ * cdef __Pyx_memviewslice memviewslice
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_1);
+ __Pyx_AddTraceback("View.MemoryView.memoryview_copy", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":1087
+ *
+ * @cname('__pyx_memoryview_copy_object_from_slice')
+ * cdef memoryview_copy_from_slice(memoryview memview, __Pyx_memviewslice *memviewslice): # <<<<<<<<<<<<<<
+ * """
+ * Create a new memoryview object from a given memoryview object and slice.
+ */
+
+static PyObject *__pyx_memoryview_copy_object_from_slice(struct __pyx_memoryview_obj *__pyx_v_memview, __Pyx_memviewslice *__pyx_v_memviewslice) {
+ PyObject *(*__pyx_v_to_object_func)(char *);
+ int (*__pyx_v_to_dtype_func)(char *, PyObject *);
+ PyObject *__pyx_r = NULL;
+ __Pyx_RefNannyDeclarations
+ int __pyx_t_1;
+ int __pyx_t_2;
+ PyObject *(*__pyx_t_3)(char *);
+ int (*__pyx_t_4)(char *, PyObject *);
+ PyObject *__pyx_t_5 = NULL;
+ int __pyx_lineno = 0;
+ const char *__pyx_filename = NULL;
+ int __pyx_clineno = 0;
+ __Pyx_RefNannySetupContext("memoryview_copy_from_slice", 0);
+
+ /* "View.MemoryView":1094
+ * cdef int (*to_dtype_func)(char *, object) except 0
+ *
+ * if isinstance(memview, _memoryviewslice): # <<<<<<<<<<<<<<
+ * to_object_func = (<_memoryviewslice> memview).to_object_func
+ * to_dtype_func = (<_memoryviewslice> memview).to_dtype_func
+ */
+ __pyx_t_1 = __Pyx_TypeCheck(((PyObject *)__pyx_v_memview), __pyx_memoryviewslice_type);
+ __pyx_t_2 = (__pyx_t_1 != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":1095
+ *
+ * if isinstance(memview, _memoryviewslice):
+ * to_object_func = (<_memoryviewslice> memview).to_object_func # <<<<<<<<<<<<<<
+ * to_dtype_func = (<_memoryviewslice> memview).to_dtype_func
+ * else:
+ */
+ __pyx_t_3 = ((struct __pyx_memoryviewslice_obj *)__pyx_v_memview)->to_object_func;
+ __pyx_v_to_object_func = __pyx_t_3;
+
+ /* "View.MemoryView":1096
+ * if isinstance(memview, _memoryviewslice):
+ * to_object_func = (<_memoryviewslice> memview).to_object_func
+ * to_dtype_func = (<_memoryviewslice> memview).to_dtype_func # <<<<<<<<<<<<<<
+ * else:
+ * to_object_func = NULL
+ */
+ __pyx_t_4 = ((struct __pyx_memoryviewslice_obj *)__pyx_v_memview)->to_dtype_func;
+ __pyx_v_to_dtype_func = __pyx_t_4;
+
+ /* "View.MemoryView":1094
+ * cdef int (*to_dtype_func)(char *, object) except 0
+ *
+ * if isinstance(memview, _memoryviewslice): # <<<<<<<<<<<<<<
+ * to_object_func = (<_memoryviewslice> memview).to_object_func
+ * to_dtype_func = (<_memoryviewslice> memview).to_dtype_func
+ */
+ goto __pyx_L3;
+ }
+
+ /* "View.MemoryView":1098
+ * to_dtype_func = (<_memoryviewslice> memview).to_dtype_func
+ * else:
+ * to_object_func = NULL # <<<<<<<<<<<<<<
+ * to_dtype_func = NULL
+ *
+ */
+ /*else*/ {
+ __pyx_v_to_object_func = NULL;
+
+ /* "View.MemoryView":1099
+ * else:
+ * to_object_func = NULL
+ * to_dtype_func = NULL # <<<<<<<<<<<<<<
+ *
+ * return memoryview_fromslice(memviewslice[0], memview.view.ndim,
+ */
+ __pyx_v_to_dtype_func = NULL;
+ }
+ __pyx_L3:;
+
+ /* "View.MemoryView":1101
+ * to_dtype_func = NULL
+ *
+ * return memoryview_fromslice(memviewslice[0], memview.view.ndim, # <<<<<<<<<<<<<<
+ * to_object_func, to_dtype_func,
+ * memview.dtype_is_object)
+ */
+ __Pyx_XDECREF(__pyx_r);
+
+ /* "View.MemoryView":1103
+ * return memoryview_fromslice(memviewslice[0], memview.view.ndim,
+ * to_object_func, to_dtype_func,
+ * memview.dtype_is_object) # <<<<<<<<<<<<<<
+ *
+ *
+ */
+ __pyx_t_5 = __pyx_memoryview_fromslice((__pyx_v_memviewslice[0]), __pyx_v_memview->view.ndim, __pyx_v_to_object_func, __pyx_v_to_dtype_func, __pyx_v_memview->dtype_is_object); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 1101, __pyx_L1_error)
+ __Pyx_GOTREF(__pyx_t_5);
+ __pyx_r = __pyx_t_5;
+ __pyx_t_5 = 0;
+ goto __pyx_L0;
+
+ /* "View.MemoryView":1087
+ *
+ * @cname('__pyx_memoryview_copy_object_from_slice')
+ * cdef memoryview_copy_from_slice(memoryview memview, __Pyx_memviewslice *memviewslice): # <<<<<<<<<<<<<<
+ * """
+ * Create a new memoryview object from a given memoryview object and slice.
+ */
+
+ /* function exit code */
+ __pyx_L1_error:;
+ __Pyx_XDECREF(__pyx_t_5);
+ __Pyx_AddTraceback("View.MemoryView.memoryview_copy_from_slice", __pyx_clineno, __pyx_lineno, __pyx_filename);
+ __pyx_r = 0;
+ __pyx_L0:;
+ __Pyx_XGIVEREF(__pyx_r);
+ __Pyx_RefNannyFinishContext();
+ return __pyx_r;
+}
+
+/* "View.MemoryView":1109
+ *
+ *
+ * cdef Py_ssize_t abs_py_ssize_t(Py_ssize_t arg) nogil: # <<<<<<<<<<<<<<
+ * if arg < 0:
+ * return -arg
+ */
+
+static Py_ssize_t abs_py_ssize_t(Py_ssize_t __pyx_v_arg) {
+ Py_ssize_t __pyx_r;
+ int __pyx_t_1;
+
+ /* "View.MemoryView":1110
+ *
+ * cdef Py_ssize_t abs_py_ssize_t(Py_ssize_t arg) nogil:
+ * if arg < 0: # <<<<<<<<<<<<<<
+ * return -arg
+ * else:
+ */
+ __pyx_t_1 = ((__pyx_v_arg < 0) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":1111
+ * cdef Py_ssize_t abs_py_ssize_t(Py_ssize_t arg) nogil:
+ * if arg < 0:
+ * return -arg # <<<<<<<<<<<<<<
+ * else:
+ * return arg
+ */
+ __pyx_r = (-__pyx_v_arg);
+ goto __pyx_L0;
+
+ /* "View.MemoryView":1110
+ *
+ * cdef Py_ssize_t abs_py_ssize_t(Py_ssize_t arg) nogil:
+ * if arg < 0: # <<<<<<<<<<<<<<
+ * return -arg
+ * else:
+ */
+ }
+
+ /* "View.MemoryView":1113
+ * return -arg
+ * else:
+ * return arg # <<<<<<<<<<<<<<
+ *
+ * @cname('__pyx_get_best_slice_order')
+ */
+ /*else*/ {
+ __pyx_r = __pyx_v_arg;
+ goto __pyx_L0;
+ }
+
+ /* "View.MemoryView":1109
+ *
+ *
+ * cdef Py_ssize_t abs_py_ssize_t(Py_ssize_t arg) nogil: # <<<<<<<<<<<<<<
+ * if arg < 0:
+ * return -arg
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ return __pyx_r;
+}
+
+/* "View.MemoryView":1116
+ *
+ * @cname('__pyx_get_best_slice_order')
+ * cdef char get_best_order(__Pyx_memviewslice *mslice, int ndim) nogil: # <<<<<<<<<<<<<<
+ * """
+ * Figure out the best memory access order for a given slice.
+ */
+
+static char __pyx_get_best_slice_order(__Pyx_memviewslice *__pyx_v_mslice, int __pyx_v_ndim) {
+ int __pyx_v_i;
+ Py_ssize_t __pyx_v_c_stride;
+ Py_ssize_t __pyx_v_f_stride;
+ char __pyx_r;
+ int __pyx_t_1;
+ int __pyx_t_2;
+ int __pyx_t_3;
+ int __pyx_t_4;
+
+ /* "View.MemoryView":1121
+ * """
+ * cdef int i
+ * cdef Py_ssize_t c_stride = 0 # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t f_stride = 0
+ *
+ */
+ __pyx_v_c_stride = 0;
+
+ /* "View.MemoryView":1122
+ * cdef int i
+ * cdef Py_ssize_t c_stride = 0
+ * cdef Py_ssize_t f_stride = 0 # <<<<<<<<<<<<<<
+ *
+ * for i in range(ndim - 1, -1, -1):
+ */
+ __pyx_v_f_stride = 0;
+
+ /* "View.MemoryView":1124
+ * cdef Py_ssize_t f_stride = 0
+ *
+ * for i in range(ndim - 1, -1, -1): # <<<<<<<<<<<<<<
+ * if mslice.shape[i] > 1:
+ * c_stride = mslice.strides[i]
+ */
+ for (__pyx_t_1 = (__pyx_v_ndim - 1); __pyx_t_1 > -1; __pyx_t_1-=1) {
+ __pyx_v_i = __pyx_t_1;
+
+ /* "View.MemoryView":1125
+ *
+ * for i in range(ndim - 1, -1, -1):
+ * if mslice.shape[i] > 1: # <<<<<<<<<<<<<<
+ * c_stride = mslice.strides[i]
+ * break
+ */
+ __pyx_t_2 = (((__pyx_v_mslice->shape[__pyx_v_i]) > 1) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":1126
+ * for i in range(ndim - 1, -1, -1):
+ * if mslice.shape[i] > 1:
+ * c_stride = mslice.strides[i] # <<<<<<<<<<<<<<
+ * break
+ *
+ */
+ __pyx_v_c_stride = (__pyx_v_mslice->strides[__pyx_v_i]);
+
+ /* "View.MemoryView":1127
+ * if mslice.shape[i] > 1:
+ * c_stride = mslice.strides[i]
+ * break # <<<<<<<<<<<<<<
+ *
+ * for i in range(ndim):
+ */
+ goto __pyx_L4_break;
+
+ /* "View.MemoryView":1125
+ *
+ * for i in range(ndim - 1, -1, -1):
+ * if mslice.shape[i] > 1: # <<<<<<<<<<<<<<
+ * c_stride = mslice.strides[i]
+ * break
+ */
+ }
+ }
+ __pyx_L4_break:;
+
+ /* "View.MemoryView":1129
+ * break
+ *
+ * for i in range(ndim): # <<<<<<<<<<<<<<
+ * if mslice.shape[i] > 1:
+ * f_stride = mslice.strides[i]
+ */
+ __pyx_t_1 = __pyx_v_ndim;
+ __pyx_t_3 = __pyx_t_1;
+ for (__pyx_t_4 = 0; __pyx_t_4 < __pyx_t_3; __pyx_t_4+=1) {
+ __pyx_v_i = __pyx_t_4;
+
+ /* "View.MemoryView":1130
+ *
+ * for i in range(ndim):
+ * if mslice.shape[i] > 1: # <<<<<<<<<<<<<<
+ * f_stride = mslice.strides[i]
+ * break
+ */
+ __pyx_t_2 = (((__pyx_v_mslice->shape[__pyx_v_i]) > 1) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":1131
+ * for i in range(ndim):
+ * if mslice.shape[i] > 1:
+ * f_stride = mslice.strides[i] # <<<<<<<<<<<<<<
+ * break
+ *
+ */
+ __pyx_v_f_stride = (__pyx_v_mslice->strides[__pyx_v_i]);
+
+ /* "View.MemoryView":1132
+ * if mslice.shape[i] > 1:
+ * f_stride = mslice.strides[i]
+ * break # <<<<<<<<<<<<<<
+ *
+ * if abs_py_ssize_t(c_stride) <= abs_py_ssize_t(f_stride):
+ */
+ goto __pyx_L7_break;
+
+ /* "View.MemoryView":1130
+ *
+ * for i in range(ndim):
+ * if mslice.shape[i] > 1: # <<<<<<<<<<<<<<
+ * f_stride = mslice.strides[i]
+ * break
+ */
+ }
+ }
+ __pyx_L7_break:;
+
+ /* "View.MemoryView":1134
+ * break
+ *
+ * if abs_py_ssize_t(c_stride) <= abs_py_ssize_t(f_stride): # <<<<<<<<<<<<<<
+ * return 'C'
+ * else:
+ */
+ __pyx_t_2 = ((abs_py_ssize_t(__pyx_v_c_stride) <= abs_py_ssize_t(__pyx_v_f_stride)) != 0);
+ if (__pyx_t_2) {
+
+ /* "View.MemoryView":1135
+ *
+ * if abs_py_ssize_t(c_stride) <= abs_py_ssize_t(f_stride):
+ * return 'C' # <<<<<<<<<<<<<<
+ * else:
+ * return 'F'
+ */
+ __pyx_r = 'C';
+ goto __pyx_L0;
+
+ /* "View.MemoryView":1134
+ * break
+ *
+ * if abs_py_ssize_t(c_stride) <= abs_py_ssize_t(f_stride): # <<<<<<<<<<<<<<
+ * return 'C'
+ * else:
+ */
+ }
+
+ /* "View.MemoryView":1137
+ * return 'C'
+ * else:
+ * return 'F' # <<<<<<<<<<<<<<
+ *
+ * @cython.cdivision(True)
+ */
+ /*else*/ {
+ __pyx_r = 'F';
+ goto __pyx_L0;
+ }
+
+ /* "View.MemoryView":1116
+ *
+ * @cname('__pyx_get_best_slice_order')
+ * cdef char get_best_order(__Pyx_memviewslice *mslice, int ndim) nogil: # <<<<<<<<<<<<<<
+ * """
+ * Figure out the best memory access order for a given slice.
+ */
+
+ /* function exit code */
+ __pyx_L0:;
+ return __pyx_r;
+}
+
+/* "View.MemoryView":1140
+ *
+ * @cython.cdivision(True)
+ * cdef void _copy_strided_to_strided(char *src_data, Py_ssize_t *src_strides, # <<<<<<<<<<<<<<
+ * char *dst_data, Py_ssize_t *dst_strides,
+ * Py_ssize_t *src_shape, Py_ssize_t *dst_shape,
+ */
+
+static void _copy_strided_to_strided(char *__pyx_v_src_data, Py_ssize_t *__pyx_v_src_strides, char *__pyx_v_dst_data, Py_ssize_t *__pyx_v_dst_strides, Py_ssize_t *__pyx_v_src_shape, Py_ssize_t *__pyx_v_dst_shape, int __pyx_v_ndim, size_t __pyx_v_itemsize) {
+ CYTHON_UNUSED Py_ssize_t __pyx_v_i;
+ CYTHON_UNUSED Py_ssize_t __pyx_v_src_extent;
+ Py_ssize_t __pyx_v_dst_extent;
+ Py_ssize_t __pyx_v_src_stride;
+ Py_ssize_t __pyx_v_dst_stride;
+ int __pyx_t_1;
+ int __pyx_t_2;
+ int __pyx_t_3;
+ Py_ssize_t __pyx_t_4;
+ Py_ssize_t __pyx_t_5;
+ Py_ssize_t __pyx_t_6;
+
+ /* "View.MemoryView":1147
+ *
+ * cdef Py_ssize_t i
+ * cdef Py_ssize_t src_extent = src_shape[0] # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t dst_extent = dst_shape[0]
+ * cdef Py_ssize_t src_stride = src_strides[0]
+ */
+ __pyx_v_src_extent = (__pyx_v_src_shape[0]);
+
+ /* "View.MemoryView":1148
+ * cdef Py_ssize_t i
+ * cdef Py_ssize_t src_extent = src_shape[0]
+ * cdef Py_ssize_t dst_extent = dst_shape[0] # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t src_stride = src_strides[0]
+ * cdef Py_ssize_t dst_stride = dst_strides[0]
+ */
+ __pyx_v_dst_extent = (__pyx_v_dst_shape[0]);
+
+ /* "View.MemoryView":1149
+ * cdef Py_ssize_t src_extent = src_shape[0]
+ * cdef Py_ssize_t dst_extent = dst_shape[0]
+ * cdef Py_ssize_t src_stride = src_strides[0] # <<<<<<<<<<<<<<
+ * cdef Py_ssize_t dst_stride = dst_strides[0]
+ *
+ */
+ __pyx_v_src_stride = (__pyx_v_src_strides[0]);
+
+ /* "View.MemoryView":1150
+ * cdef Py_ssize_t dst_extent = dst_shape[0]
+ * cdef Py_ssize_t src_stride = src_strides[0]
+ * cdef Py_ssize_t dst_stride = dst_strides[0] # <<<<<<<<<<<<<<
+ *
+ * if ndim == 1:
+ */
+ __pyx_v_dst_stride = (__pyx_v_dst_strides[0]);
+
+ /* "View.MemoryView":1152
+ * cdef Py_ssize_t dst_stride = dst_strides[0]
+ *
+ * if ndim == 1: # <<<<<<<<<<<<<<
+ * if (src_stride > 0 and dst_stride > 0 and
+ * src_stride == itemsize == dst_stride):
+ */
+ __pyx_t_1 = ((__pyx_v_ndim == 1) != 0);
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":1153
+ *
+ * if ndim == 1:
+ * if (src_stride > 0 and dst_stride > 0 and # <<<<<<<<<<<<<<
+ * src_stride == itemsize == dst_stride):
+ * memcpy(dst_data, src_data, itemsize * dst_extent)
+ */
+ __pyx_t_2 = ((__pyx_v_src_stride > 0) != 0);
+ if (__pyx_t_2) {
+ } else {
+ __pyx_t_1 = __pyx_t_2;
+ goto __pyx_L5_bool_binop_done;
+ }
+ __pyx_t_2 = ((__pyx_v_dst_stride > 0) != 0);
+ if (__pyx_t_2) {
+ } else {
+ __pyx_t_1 = __pyx_t_2;
+ goto __pyx_L5_bool_binop_done;
+ }
+
+ /* "View.MemoryView":1154
+ * if ndim == 1:
+ * if (src_stride > 0 and dst_stride > 0 and
+ * src_stride == itemsize == dst_stride): # <<<<<<<<<<<<<<
+ * memcpy(dst_data, src_data, itemsize * dst_extent)
+ * else:
+ */
+ __pyx_t_2 = (((size_t)__pyx_v_src_stride) == __pyx_v_itemsize);
+ if (__pyx_t_2) {
+ __pyx_t_2 = (__pyx_v_itemsize == ((size_t)__pyx_v_dst_stride));
+ }
+ __pyx_t_3 = (__pyx_t_2 != 0);
+ __pyx_t_1 = __pyx_t_3;
+ __pyx_L5_bool_binop_done:;
+
+ /* "View.MemoryView":1153
+ *
+ * if ndim == 1:
+ * if (src_stride > 0 and dst_stride > 0 and # <<<<<<<<<<<<<<
+ * src_stride == itemsize == dst_stride):
+ * memcpy(dst_data, src_data, itemsize * dst_extent)
+ */
+ if (__pyx_t_1) {
+
+ /* "View.MemoryView":1155
+ * if (src_stride > 0 and dst_stride > 0 and
+ * src_stride == itemsize == dst_stride):
+ * memcpy(dst_data, src_data, itemsize * dst_extent) # <<<<<<<<<<<<<<
+ * else:
+ * for i in range(dst_extent):
+ */
+ (void)(memcpy(__pyx_v_dst_data, __pyx_v_src_data, (__pyx_v_itemsize * __pyx_v_dst_extent)));
+
+ /* "View.MemoryView":1153
+ *
+ * if ndim == 1:
+ * if (src_stride > 0 and dst_stride > 0 and # <<<<<<<<<<<<<<
+ *