diff --git a/docs/ai/computer-vision-cv/convolutional-neural-network-cnn.md b/docs/ai/computer-vision-cv/convolutional-neural-network-cnn.md new file mode 100644 index 00000000000..b149eb03c4f --- /dev/null +++ b/docs/ai/computer-vision-cv/convolutional-neural-network-cnn.md @@ -0,0 +1,37 @@ +# Convolutional Neural Network (CNN) + +## Neural Networks + +Among [deep neural networks (DNN)](https://viso.ai/deep-learning/deep-neural-network-three-popular-types/), the [convolutional neural network (CNN)](https://viso.ai/deep-learning/convolutional-neural-networks/) has demonstrated excellent results in computer vision tasks, especially in image classification. Convolutional Neural Networks (CNNs) are a special type of multi-layer neural network inspired by the mechanism of human optical and neural systems. + +In 2012, a large deep convolutional neural network called [AlexNet](https://viso.ai/deep-learning/alexnet/) showed excellent performance on the ImageNet Large Scale Visual Recognition Challenge (ILSVRC). This marked the start of the broad use and development of convolutional neural network models (CNN) such as [VGGNet](https://viso.ai/deep-learning/vgg-very-deep-convolutional-networks/), [GoogleNet](https://viso.ai/deep-learning/googlenet-explained-the-inception-model-that-won-imagenet/), [ResNet](https://viso.ai/deep-learning/resnet-residual-neural-network/), DenseNet, and many more. + +## Convolutional Neural Network (CNN) + +A CNN is a framework developed using machine learning concepts. CNNs can learn and train from data on their own without the need for human intervention. + +There is only some pre-processing needed when using CNNs. They develop and adapt their image filters, which have to be carefully coded for most algorithms and models. CNN frameworks have a set of layers that perform particular functions to enable CNN to perform these functions. + +## CNN Architecture + +The basic unit of a CNN framework is a neuron. The concept of neurons is based on human neurons, where synapses occur due to [neuron activation](https://viso.ai/deep-learning/neuron-activation/). These are statistical functions that calculate the weighted average of inputs and apply an activation function to the result generated. Layers are a cluster of neurons, with each layer having a particular function. + + +![Concept of a neural network](../../media/Pasted%20image%2020240917123040.png) + +## CNN Layers + +A CNN system may have somewhere between 3 to 150 or even more layers: The “deep” of Deep neural networks refers to the number of layers. One layer’s output acts as another layer’s input. Deep multi-layer neural networks include [Resnet50 (50 layers) or ResNet101 (101 layers)](https://viso.ai/deep-learning/resnet-residual-neural-network/). + +![Concept of a Convolutional Neural Network (CNN)](../../media/Pasted%20image%2020240917123109.png) + +CNN layers can be of four main types: Convolution Layer, ReLu Layer, Pooling Layer, and Fully-Connected Layer. + +- **Convolution Layer:** A convolution is the simple application of a filter to an input that results in an activation. The convolution layer has a set of trainable filters that have a small receptive range but can be used to the full depth of data provided. Convolution layers are the major building blocks used in convolutional neural networks. +- **ReLu Layer:** ReLu layers, or Rectified linear unit layers, are activation functions for lowering [overfitting](https://viso.ai/computer-vision/what-is-overfitting/) and building CNN accuracy and effectiveness. Models that have these layers are easier to train and produce more accurate results. +- **Pooling Layer:** This layer collects the result of all neurons in the layer preceding it and processes this data. The primary task of a pooling layer is to lower the number of considered factors and give streamlined output. +- **Fully-Connected Layer:** This layer is the final output layer for CNN models that flattens the input data received from layers before it and gives the result. + +## Links + +- [A Complete Guide to Image Classification in 2024 - viso.ai](https://viso.ai/computer-vision/image-classification/) diff --git a/docs/ai/computer-vision-cv/intro.md b/docs/ai/computer-vision-cv/intro.md index f434b66b847..365ebe8c520 100755 --- a/docs/ai/computer-vision-cv/intro.md +++ b/docs/ai/computer-vision-cv/intro.md @@ -82,12 +82,8 @@ https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio ## References -[Self Driving Nanodegree](courses/self-driving-nanodegree.md) - -https://towardsdatascience.com/understanding-ssd-multibox-real-time-object-detection-in-deep-learning-495ef744fab - -https://www.freecodecamp.org/news/advanced-computer-vision-with-python - -[Comic book panel segmentation • Max Halford](https://maxhalford.github.io/blog/comic-book-panel-segmentation/) - -[Unbelievable Face Swapping with 5 Lines Code - YouTube](https://www.youtube.com/watch?v=a8vFMaH2aDw) +- [Self Driving Nanodegree](courses/self-driving-nanodegree.md) +- https://towardsdatascience.com/understanding-ssd-multibox-real-time-object-detection-in-deep-learning-495ef744fab +- https://www.freecodecamp.org/news/advanced-computer-vision-with-python +- [Comic book panel segmentation • Max Halford](https://maxhalford.github.io/blog/comic-book-panel-segmentation/) +- [Unbelievable Face Swapping with 5 Lines Code - YouTube](https://www.youtube.com/watch?v=a8vFMaH2aDw) diff --git a/docs/ai/computer-vision-cv/model-building-stages.md b/docs/ai/computer-vision-cv/model-building-stages.md new file mode 100644 index 00000000000..b2ac2b085ff --- /dev/null +++ b/docs/ai/computer-vision-cv/model-building-stages.md @@ -0,0 +1,96 @@ +# Model Building Stages + +## 1. Define the Problem + +Clearly define the goal of the project: **to build a CV model that detects aflatoxin contamination levels in corn samples through image analysis**. The contamination levels will be categorized into predefined bands such as 0-30 ppb, 31-50 ppb, etc. + +- **Output**: Classification of aflatoxin levels into one of the specified categories. +- **Performance Target**: Achieve at least 80% accuracy in classifying contamination levels. + +## 2. Collect and Label Data + +The success of a CV model depends heavily on the quality and quantity of data: + +- **Image Dataset**: Obtain a dataset of corn images provided by the client, with images labeled based on the aflatoxin contamination levels. +- **Data Labels**: Ensure that each image has a label that specifies the contamination level (in ppb). These will serve as the ground truth for training the model. +- **Data Size**: Ensure the dataset is large enough to prevent overfitting. If the dataset is small, consider techniques like **data augmentation** to artificially increase the dataset size. + +## 3. Preprocess the Data + +Preprocessing the images is essential to standardize the input data for the model: + +- **Normalization**: Scale pixel values to a range of `[0, 1]` or `[-1, 1]` to help the model converge faster. +- **Resizing**: Resize all images to a fixed resolution (e.g., 224x224 pixels) to ensure consistency in input size. +- **Augmentation**: Apply image augmentation techniques (e.g., rotation, flipping, zoom, brightness adjustments) to make the model more robust to variations in real-world conditions. +- **Train-Validation Split**: Split the dataset into training and validation sets (e.g., 80% training, 20% validation) to evaluate model performance during development. + +## 4. Choose a Model Architecture + +For image classification tasks, **Convolutional Neural Networks (CNNs)** are the most commonly used architectures: + +- **Pre-trained Models (Transfer Learning)**: + - Use pre-trained models like **ResNet**, **MobileNet**, or **EfficientNet** to leverage knowledge from large datasets like ImageNet. This can reduce training time and improve accuracy. + - **Transfer Learning**: Fine-tune the pre-trained model on your specific dataset by replacing the final layer(s) to output the aflatoxin contamination categories. +- **Custom CNN Architecture**: + - If transfer learning isn’t sufficient, a custom CNN architecture can be built. Design layers that fit the complexity of your data, including convolutional layers, pooling layers, and fully connected layers. + +## 5. Train the Model + +Now that the data is prepared and the model architecture is selected, proceed to training: + +- **Loss Function**: Use **categorical cross-entropy** as the loss function since this is a multi-class classification problem. +- **Optimizer**: Use optimizers like **Adam** or **SGD** with momentum to adjust learning rates and improve convergence. +- **Batch Size & Epochs**: Experiment with different batch sizes (e.g., 32, 64) and run multiple epochs (e.g., 50-100 epochs). Monitor overfitting using early stopping techniques. +- **Hyperparameter Tuning**: Fine-tune hyperparameters like learning rate, dropout rate, and number of layers to optimize performance. + +## 6. Evaluate the Model + +After training, evaluate the model to ensure it meets the desired performance criteria: + +- **Confusion Matrix**: Generate a confusion matrix to analyze how well the model performs across all contamination bands (e.g., 0-30 ppb, 31-50 ppb). +- **Performance Metrics**: Evaluate key metrics like accuracy, precision, recall, F1-score for each class. For imbalanced datasets, consider using **weighted precision/recall**. +- **Cross-Validation**: Perform **k-fold cross-validation** to ensure that the model generalizes well across different subsets of the data. + +## 7. Improve the Model + +If the model does not meet the performance goals, several techniques can be used to improve it: + +- **Data Augmentation**: Further enhance the dataset by introducing more variability in the training data. +- **Model Regularization**: Use techniques like **dropout**, **batch normalization**, or **L2 regularization** to prevent overfitting. +- **Hyperparameter Tuning**: Use methods like **grid search** or **random search** to find optimal values for hyperparameters (e.g., learning rate, batch size). +- **Ensemble Methods**: Combine multiple models (e.g., bagging or boosting) to improve prediction accuracy. + +## 8. Test the Model + +Once the model is fine-tuned and evaluated, test its performance on a **holdout test set** or new data provided by the client: + +- **Validation on New Data**: Use unseen images from the client’s dataset to ensure that the model generalizes well to real-world samples. +- **Performance Metrics Report**: Document the model's final accuracy, confusion matrix, and other performance metrics. + +## 9. Deploy the Model (For POC) + +For the POC phase, the model will be deployed in a hosted environment (cloud or on-prem): + +- **Deploy on Vendor's Environment**: Host the model on a cloud server (e.g., AWS, Azure) where it can accept image inputs and return aflatoxin contamination levels via an API. +- **Performance Monitoring**: Set up tools to monitor inference time, model accuracy, and resource utilization to ensure smooth operation. + +## 10. Document and Report Results + +After deployment, prepare a comprehensive report to present to the client: + +- **POC Results**: Include detailed results of the model’s performance (e.g., accuracy, confusion matrix). +- **Recommendations for Future Phases**: Provide insights on how the model can be scaled and improved further in Phase 2 (e.g., mobile app integration, on-device inference). + +## Tools and Technologies for Each Step + +1. **Preprocessing & Data Augmentation**: + - Tools: **OpenCV**, **Keras ImageDataGenerator**, **Albumentations** +2. **Model Development**: + - Tools: **TensorFlow**, **Keras**, **PyTorch** (for building CNNs and transfer learning) +3. **Training & Optimization**: + - Optimizers: **Adam**, **SGD** + - Techniques: Early stopping, learning rate scheduling +4. **Evaluation**: + - Tools: **scikit-learn** (for confusion matrices and performance metrics) +5. **Deployment**: + - Tools: **AWS SageMaker**, **Azure ML**, or **Google AI Platform** diff --git a/docs/ai/computer-vision-cv/pre-trained-models.md b/docs/ai/computer-vision-cv/pre-trained-models.md index c8fa79b6ce4..8937f89f620 100644 --- a/docs/ai/computer-vision-cv/pre-trained-models.md +++ b/docs/ai/computer-vision-cv/pre-trained-models.md @@ -109,7 +109,7 @@ Here's how YOLO works: - **Grid:** YOLO's CNN divides an image into a grid. - **Bounding boxes:** Each cell in the grid predicts a number of bounding boxes. -- **Class probabilities:** Each cell also predicts a class probability, which indicates the likelihood of an object being present in the box.  +- **Class probabilities:** Each cell also predicts a class probability, which indicates the likelihood of an object being present in the box. YOLO is popular because of its single-stage architecture, real-time performance, and accuracy. It's well-suited for real-time applications like self-driving cars, video surveillance, and augmented reality. @@ -139,6 +139,12 @@ YOLO is popular because of its single-stage architecture, real-time performance, | BiT-L (ResNet) | **928 M** | 87.54 % | 2019 | | NoisyStudent EfficientNet-L2 | **480 M** | **88.4** % | 2020 | | Meta Pseudo Labels | **480 M** | **90.2** % | 2021 | +| CoCa (finetuned) | 2100M | 91.0% | 2022 | +| OmniVec (ViT) | | 92.4% | 2023 | + +Leaderboard - [ImageNet Benchmark (Image Classification) | Papers With Code](https://paperswithcode.com/sota/image-classification-on-imagenet) + +Models - [Models - Hugging Face](https://huggingface.co/models?pipeline_tag=image-classification) - [CNN Architectures: LeNet, AlexNet, VGG, GoogLeNet, ResNet and more… | by Siddharth Das | Analytics Vidhya | Medium](https://medium.com/analytics-vidhya/cnns-architectures-lenet-alexnet-vgg-googlenet-resnet-and-more-666091488df5) - [Difference between AlexNet, VGGNet, ResNet, and Inception | by Aqeel Anwar | Towards Data Science](https://towardsdatascience.com/the-w3h-of-alexnet-vggnet-resnet-and-inception-7baaaecccc96) @@ -160,3 +166,4 @@ YOLO is popular because of its single-stage architecture, real-time performance, - [Top Pre-Trained Models for Image Classification - GeeksforGeeks](https://www.geeksforgeeks.org/top-pre-trained-models-for-image-classification/) - [Top 4 Pre-Trained Models for Image Classification + Python Code](https://www.analyticsvidhya.com/blog/2020/08/top-4-pre-trained-models-for-image-classification-with-python-code/) - [Best deep CNN architectures and their principles: from AlexNet to EfficientNet | AI Summer](https://theaisummer.com/cnn-architectures/) +- [7 Best Image Classification Models You Should Know in 2023 - Jonas Cleveland](https://jonascleveland.com/best-image-classification-models/) diff --git a/docs/ai/computer-vision-cv/readme.md b/docs/ai/computer-vision-cv/readme.md index 0138d097db0..e116690a5be 100755 --- a/docs/ai/computer-vision-cv/readme.md +++ b/docs/ai/computer-vision-cv/readme.md @@ -2,6 +2,8 @@ - [Computer Vision (CV) Intro](ai/computer-vision-cv/intro.md) - [Pre-Trained Models](ai/computer-vision-cv/pre-trained-models.md) +- [Convolutional Neural Network (CNN)](ai/computer-vision-cv/convolutional-neural-network-cnn.md) +- [Model Building Stages](ai/computer-vision-cv/model-building-stages.md) - [Image / Data Labeling Tools](image-data-labeling-tools) - [Image Formats](image-formats) - [MNIST for ML Begineers | Tensorflow](mnist-for-ml-beginners-tensorflow) diff --git a/docs/ai/data-science/datasets.md b/docs/ai/data-science/datasets.md index 5f37b7d0e6d..bc1c58aa50d 100755 --- a/docs/ai/data-science/datasets.md +++ b/docs/ai/data-science/datasets.md @@ -1,5 +1,7 @@ # Datasets +[Home - Data Commons](https://datacommons.org/) + https://www.kaggle.com/dalpozz/creditcardfraud [20+ Amazing (And Free) Data Sources Anyone Can Use To Build AIs](https://www.forbes.com/sites/bernardmarr/2023/05/17/20-amazing-and-free-data-sources-anyone-can-use-to-build-ais/?sh=17c13eec617f) diff --git a/docs/ai/deep-learning/commands.md b/docs/ai/deep-learning/commands.md index 1e83f102320..c5ee5d202c8 100755 --- a/docs/ai/deep-learning/commands.md +++ b/docs/ai/deep-learning/commands.md @@ -63,7 +63,7 @@ def L1(yhat, y): def L2(yhat, y): loss = np.sum(np.dot((y-yhat),(y-yhat))) -A trick when you want to flatten a matrix X of shape (a, b, c, d) to a matrix X_flatten of shape (b∗∗c∗∗d, a) is to use: +# A trick when you want to flatten a matrix X of shape (a, b, c, d) to a matrix X_flatten of shape (b∗∗c∗∗d, a) is to use: X_flatten = X.reshape(X.shape [0], -1).T ``` diff --git a/docs/ai/deep-learning/dl-specialization.md b/docs/ai/deep-learning/dl-specialization.md index 11af82b4faa..bd5779fd8c7 100755 --- a/docs/ai/deep-learning/dl-specialization.md +++ b/docs/ai/deep-learning/dl-specialization.md @@ -2,9 +2,15 @@ [**https://www.coursera.org/specializations/deep-learning**](https://www.coursera.org/specializations/deep-learning) -1. **Neural Networks and Deep Learning** +## 1. Neural Networks and Deep Learning -Understand the major technology trends driving Deep Learning - Be able to build, train and apply fully connected deep neural networks - Know how to implement efficient (vectorized) neural networks - Understand the key parameters in a neural network's architecture This course also teaches you how Deep Learning actually works, rather than presenting only a cursory or surface-level description. +Understand the major technology trends driving Deep Learning + +- Be able to build, train and apply fully connected deep neural networks +- Know how to implement efficient (vectorized) neural networks +- Understand the key parameters in a neural network's architecture + +This course also teaches you how Deep Learning actually works, rather than presenting only a cursory or surface-level description. Week 1 - Introduction to deep learning @@ -21,27 +27,47 @@ Week 3 - Shallow neural networks Week 4 - Deep neural networks -2. **Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization** +## 2. Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization This course will teach you the "magic" of getting deep learning to work well. Rather than the deep learning process being a black box, you will understand what drives performance, and be able to more systematically get good results. You will also learn TensorFlow. -After 3 weeks, you will: - Understand industry best-practices for building deep learning applications. - Be able to effectively use the common neural network "tricks", including initialization, L2 and dropout regularization, Batch normalization, gradient checking, - Be able to implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence. - Understand new best-practices for the deep learning era of how to set up train/dev/test sets and analyze bias/variance - Be able to implement a neural network in TensorFlow. +After 3 weeks, you will: + +- Understand industry best-practices for building deep learning applications. +- Be able to effectively use the common neural network "tricks", including initialization, L2 and dropout regularization, Batch normalization, gradient checking, +- Be able to implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence. +- Understand new best-practices for the deep learning era of how to set up train/dev/test sets and analyze bias/variance +- Be able to implement a neural network in TensorFlow. -3. **Structuring Machine Learning Projects** +## 3. Structuring Machine Learning Projects -After 2 weeks, you will: - Understand how to diagnose errors in a machine learning system, and - Be able to prioritize the most promising directions for reducing error - Understand complex ML settings, such as mismatched training/test sets, and comparing to and/or surpassing human-level performance - Know how to apply end-to-end learning, transfer learning, and multitask learning I've seen teams waste months or years through not understanding the principles taught in this course. I hope this two week course will save you months of time. This is a standalone course, and you can take this so long as you have basic machine learning knowledge. +After 2 weeks, you will: -4. **Convolutional Neural Networks (CNN)** +- Understand how to diagnose errors in a machine learning system, and +- Be able to prioritize the most promising directions for reducing error +- Understand complex ML settings, such as mismatched training/test sets, and comparing to and/or surpassing human-level performance +- Know how to apply end-to-end learning, transfer learning, and multitask learning I've seen teams waste months or years through not understanding the principles taught in this course. + +I hope this two week course will save you months of time. This is a standalone course, and you can take this so long as you have basic machine learning knowledge. + +## 4. Convolutional Neural Networks (CNN) This course will teach you how to build convolutional neural networks and apply it to image data. Thanks to deep learning, computer vision is working far better than just two years ago, and this is enabling numerous exciting applications ranging from safe autonomous driving, to accurate face recognition, to automatic reading of radiology images. -You will: - Understand how to build a convolutional neural network, including recent variations such as residual networks. - Know how to apply convolutional networks to visual detection and recognition tasks. - Know to use neural style transfer to generate art. - Be able to apply these algorithms to a variety of image, video, and other 2D or 3D data. This is the fourth course of the Deep Learning Specialization. +- Understand how to build a convolutional neural network, including recent variations such as residual networks. +- Know how to apply convolutional networks to visual detection and recognition tasks. +- Know to use neural style transfer to generate art. +- Be able to apply these algorithms to a variety of image, video, and other 2D or 3D data. + +This is the fourth course of the Deep Learning Specialization. -5. **Sequence Models** +## 5. Sequence Models This course will teach you how to build models for natural language, audio, and other sequence data. Thanks to deep learning, sequence algorithms are working far better than just two years ago, and this is enabling numerous exciting applications in speech recognition, music synthesis, chatbots, machine translation, natural language understanding, and many others. -You will: - Understand how to build and train Recurrent Neural Networks (RNNs), and commonly-used variants such as GRUs and LSTMs. - Be able to apply sequence models to natural language problems, including text synthesis. - Be able to apply sequence models to audio applications, including speech recognition and music synthesis. +- Understand how to build and train Recurrent Neural Networks (RNNs), and commonly-used variants such as GRUs and LSTMs. +- Be able to apply sequence models to natural language problems, including text synthesis. +- Be able to apply sequence models to audio applications, including speech recognition and music synthesis. ## Notes diff --git a/docs/ai/deep-learning/neural-network-and-deep-learning.md b/docs/ai/deep-learning/neural-network-and-deep-learning.md index d89ca5be2ba..9f2f4a1f008 100755 --- a/docs/ai/deep-learning/neural-network-and-deep-learning.md +++ b/docs/ai/deep-learning/neural-network-and-deep-learning.md @@ -100,7 +100,7 @@ So, it turns out that there are a set of techniques called vectorizationtechniqu ![image](../../media/DL-Specialization_Neural-network-and-deep-learning-image17.jpg) -## Z = W^T^X + b +`Z = W^T^X + b` ## Example @@ -148,7 +148,7 @@ Whenever possible, avoid explicit for-loops ![image](../../media/DL-Specialization_Neural-network-and-deep-learning-image21.jpg) -## Size of matrix X - (n~x~ , m) +`Size of matrix X - (n~x~ , m)` ## Vectorizing Logistic Regression's Gradient Output @@ -158,7 +158,7 @@ Whenever possible, avoid explicit for-loops How do you compute the derivative of *b* in one line of code in Python numpy? -1 / m*(np.sum(dz)) +`1 / m*(np.sum(dz))` ## Broadcasting in Python @@ -166,11 +166,11 @@ How do you compute the derivative of *b* in one line of code in Python numpy? Calculate % of calories from carb, protein, fat, without explicit for loop? +``` cal = A.sum(axis = 0) - percentage = 100*A / (cal.reshape(1, 4)) - matrix (3, 4) / matrix (1, 4) +``` ![image](../../media/DL-Specialization_Neural-network-and-deep-learning-image25.jpg) @@ -189,7 +189,7 @@ True or False: Minimizing the loss corresponds with maximizing logp(y|x) - **Tru - We rarely use the "math" library in deep learning because the inputs of the functions are real numbers. In deep learning we mostly use matrices and vectors. This is why numpy is more useful. - sigmoid_grad() to compute the gradient of the sigmoid function with respect to its input x. The formula is: -sigmoid_derivative(x)=σ′(x)=σ(x)(1−σ(x)) +`sigmoid_derivative(x)=σ′(x)=σ(x)(1−σ(x))` - Another common technique we use in Machine Learning and Deep Learning is to normalize our data. It often leads to a better performance because gradient descent converges faster after normalization. Here, by normalization we mean changing x tox/∥x∥(dividing each row vector of x by its norm). diff --git a/docs/ai/llm/genai-projects.md b/docs/ai/llm/genai-projects.md index 6d98cdf046b..f31e1e0698d 100644 --- a/docs/ai/llm/genai-projects.md +++ b/docs/ai/llm/genai-projects.md @@ -11,9 +11,9 @@ ### Develop your own ChatGPT from scratch - Implement all stages in the development of a Large Language Model (LLM) similar to ChatGPT, including: - - Pretraining - - Supervised Finetuning - - Reinforcement Learning from Human Feedback (RLHF) + - Pretraining + - Supervised Finetuning + - Reinforcement Learning from Human Feedback (RLHF) - Learn and apply the best industry practices for creating a dialogue- optimized LLM like ChatGPT. ### Develop your own LLM Application using Prompt Engineering @@ -25,11 +25,11 @@ - Use LlamaIndex to build a production-ready Retrieval-Augmented Generation (RAG) system on your private data. - Acquire hands-on experience with advanced components of LlamaIndex, including: - - Agents - - Router Query Engine, SubQuestionQuery Engine - - Open AI Assistants - - Finetuning with Retrieval Augmentation (RADIT) - - Finetuning Embeddings + - Agents + - Router Query Engine, SubQuestionQuery Engine + - Open AI Assistants + - Finetuning with Retrieval Augmentation (RADIT) + - Finetuning Embeddings ### Building End-to-End RAG Apps diff --git a/docs/ai/llm/intro.md b/docs/ai/llm/intro.md index ce2c335fc5f..df22eb27575 100644 --- a/docs/ai/llm/intro.md +++ b/docs/ai/llm/intro.md @@ -79,5 +79,5 @@ The architecture is a standard [transformer network](https://en.wikipedia.org/wi - [GenAI Training In Production: Software, Hardware & Network Considerations - YouTube](https://www.youtube.com/watch?v=1lhrGRqqPWU) - [How I Use "AI"](https://nicholas.carlini.com/writing/2024/how-i-use-ai.html) - [How might LLMs store facts | Chapter 7, Deep Learning - YouTube](https://www.youtube.com/watch?v=9-Jl0dxWQs8) -- [Introduction to Large Language Models - YouTube](https://www.youtube.com/watch?v=RBzXsQHjptQ) - [Introduction to Generative AI - YouTube](https://www.youtube.com/watch?v=cZaNf2rA30k) +- [Introduction to Large Language Models - YouTube](https://www.youtube.com/watch?v=RBzXsQHjptQ) diff --git a/docs/ai/llm/llm-building.md b/docs/ai/llm/llm-building.md index 91989b58374..5e4ac1934cb 100644 --- a/docs/ai/llm/llm-building.md +++ b/docs/ai/llm/llm-building.md @@ -8,6 +8,7 @@ - [Llama - EXPLAINED! - YouTube](https://www.youtube.com/watch?v=yZ9jkgN2xHQ) - [LLM2 Module 1 - Transformers | 1.6 Base/Foundation Models - YouTube](https://www.youtube.com/watch?v=sJsPgRg883w) - [20 papers to master Language modeling? - YouTube](https://www.youtube.com/watch?v=QQIwfpOY-qA) +- [Bringing Llama 3 to Life | Joe Spisak, Delia David, Kaushik Veeraraghavan & Ye (Charlotte) Qi - YouTube](https://www.youtube.com/watch?v=ELIcy6flgQI) ## Architecture diff --git a/docs/ai/llm/llm-tuning.md b/docs/ai/llm/llm-tuning.md new file mode 100644 index 00000000000..c5095aaedcb --- /dev/null +++ b/docs/ai/llm/llm-tuning.md @@ -0,0 +1,19 @@ +# LLM Tuning + +The process of adapting a model to a new domain or set of custom use cases by training the model on new data + +## Fine Tuning + +Large language model (LLM) fine-tuning is the process of taking pre-trained models and further training them on smaller, specific datasets to refine their capabilities and improve performance in a particular task or domain. Fine-tuning is about turning general-purpose models and turning them into specialized models. It bridges the gap between generic pre-trained models and the unique requirements of specific applications, ensuring that the language model aligns closely with human expectations. + +## Supervised fine-tuning (SFT) + +Supervised fine-tuning means updating a pre-trained language model using labeled data to do a specific task. The data used has been checked earlier. This is different from unsupervised methods, where data isn't checked. Usually, the initial training of the language model is unsupervised, but fine-tuning is supervised. + +## Methods for fine-tuning LLMs + +- Instruction fine-tuning +- Full fine-tuning +- Parameter-efficient fine-tuning (PEFT) + +[Fine-tuning large language models (LLMs) in 2024 | SuperAnnotate](https://www.superannotate.com/blog/llm-fine-tuning) diff --git a/docs/ai/llm/models.md b/docs/ai/llm/models.md index 347a8d733b5..1bfa1f59040 100644 --- a/docs/ai/llm/models.md +++ b/docs/ai/llm/models.md @@ -9,6 +9,14 @@ - They can discriminate between different kinds of data instances. - Trained on a dataset of images of cats and dogs and then used to classify new images as either cats or dogs. +### Types + +- **Generic or raw language models** predict the next word based on the language in the training data. These language models perform information retrieval tasks. + - The cat sat on ___ (answer - the) +- **Instruction-tuned language models** are trained to predict responses to the instructions given in the input. This allows them to perform sentiment analysis, or to generate text or code. + - Generate a poem in the style of x +- **Dialog-tuned language models** are trained to have a dialog by predicting the next response. Think of chatbots or conversational AI. + ## Models - ChatGPT / OpenAI diff --git a/docs/ai/llm/prompt-engineering.md b/docs/ai/llm/prompt-engineering.md index 9bd9d90bb43..c2c76cced1e 100644 --- a/docs/ai/llm/prompt-engineering.md +++ b/docs/ai/llm/prompt-engineering.md @@ -1,5 +1,9 @@ # Prompt Engineering +**Prompt design** is the process of creating a prompt that is tailored to the specific task that the system is being asked to perform. + +**Prompt engineering** is the process of creating a prompt that is designed to improve performance. + ## Prompting Principles ### Principle 1: Write clear and specific instructions diff --git a/docs/ai/llm/readme.md b/docs/ai/llm/readme.md index ffd712cadf2..e6306ece013 100644 --- a/docs/ai/llm/readme.md +++ b/docs/ai/llm/readme.md @@ -4,6 +4,7 @@ - [Models](ai/llm/models.md) - [Limitations / Problems](ai/llm/limitations-problems.md) - [LLM Building](ai/llm/llm-building.md) +- [LLM Tuning](ai/llm/llm-tuning.md) - [Design Patterns](ai/llm/design-patterns.md) - [RAG Retrieval Augmented Generation](ai/llm/rag-retrieval-augmented-generation.md) - [Natural Language to SQL](ai/llm/natural-language-to-sql.md) @@ -25,3 +26,5 @@ ## GenAI Landscape ![GenAI Landscape](../../media/Pasted%20image%2020240915133426.jpg) + +![GenAI Application Landscape](../../media/Screenshot%202024-09-17%20at%201.46.23%20AM.jpg) diff --git a/docs/book-summaries/books-blinkist-productivity.md b/docs/book-summaries/books-blinkist-productivity.md new file mode 100644 index 00000000000..dff7dea5809 --- /dev/null +++ b/docs/book-summaries/books-blinkist-productivity.md @@ -0,0 +1,75 @@ +# Books / Blinkist - Productivity + +## Blinkist - Get your shit together by Sarah Knight (2 Jan 2019) + +![image](../media/Productivity-Productive-Time-Management-TODO-image2.jpg) + +## Blinkist - 5 Gears: How to Be Present and Productive When There is Never Enough Time by Jeremie Kubicek, Steve Cockram (7 Jan 2019) + +- **First gear -** when you fully rest and recharge (Learning to recharge) +- **Second gear -** when you connect with family or friends without the involvement of work (Connecting deeply) +- **Third gear -** when you are socializing (Why being social matters) +- **Fourth gear -** when you are working and multitasking (Leading in the task world) +- **Fifth gear -** when you are fully focused and 'in the zone,' working without interruption (In the Zone) +- **Reverse -** Being responsive in a resistant world + +## Blinkist - Great at Work by Morten T. Hansen + +How top performers do less, work better, and achieve more + +- Mastering your own work + - Do less, then obsess + - Choosing a few priorities + - Dedicating your efforts toward excelling at them + - Redesign your work + - Start with a value not with a goal + - Don't just learn, loop + - Micro-behavior is a small, concrete action you take on a daily basis to improve a skill. The action shouldn't take more than fifteen minutes to perform and review, and it should have a clear impact on skill development. + - P-squared (Passion and Purpose) +- Mastering working with others + - Forceful champions + - Fight and unite + - Fight - On a meeting, maximize a team debate + - Unite - Commit to decision at the end of meeting, because best idea wins + - The two sins of collaboration +- Master your work-life + +## 168 Hours By Laura Vanderkam + +You Have More Time Than You Think + +- You're not too busy -- you're just not in control of your time. + - Instead of thinking, "I'm too busy," you should ask, "Is that a priority?" If it isn't, simply let it go. +- To live a fulfilling life, focus on your core competencies. +- Choosing a job that uses your core competencies energizes your entire life. + - "Any 'work' that is not advancing you towards the professional life you want should not count as work. It is wasted time." +- To become time-rich you must control your work calendar. + - Create habit of excusing yourself from meetings. If you realize a meeting isn't relevant to you, and it's a waste of your time, you should leave. And that time would be better spent pursuing activities that advance your career. +- To achieve a career breakthrough, combine strategy with story. + - "There's a lot of randomness in the universe, but truly lucky people recognize that fairy godmothers are lazy." +- You can improve your relationships with your children by focusing on your core competencies. + - If you both love swimming, schedule regular trips to the local pool. A love of music might mean singing together. +- Outsource housework that doesn't align with your core competencies. +- Plan your leisure time to increase its value. +- Actionable advice: Win family time by creating a mini night shift. + - Working parents often find it difficult to spend time with their children during the week. Overcome this by blocking out 5:00 p.m. to 8:00 p.m. in your calendar every weeknight -- and view this as sacred family time. Counter your shorter day at the office by working from 8:00 p.m. to 10:00 p.m. a few nights a week. By doing this, you'll maximize your time together while the kids are still awake. Otherwise, you're more likely to waste your evening leisure time in front of the TV. + +## The Lazy Genius Way by Kendra Adachi + +Embrace what matters, ditch what doesn't, and get stuff done + +- What's in it for me? Learn how to focus on the things that actually matter +- Being a perfectionist is exhausting and lonely -- but there is another way +- Real transformation happens in small steps +- Making fixed decisions can bring calm and focus +- You can defuse stressful situations with a little planning +- Introducing a few simple rules can transform your home +- In order to discover what's important to you, you'll need to take some time out +- We can't escape difficult situations, but we can change how we respond to them +- We need to become our own best friends -- and then allow other people in + +## Blinkist - The 12 Week Year by Brian P. Moran + +Get more done in 12 weeks than others do in 12 months + +Plan around a 12-week period, execute tasks and radically improve results diff --git a/docs/book-summaries/readme.md b/docs/book-summaries/readme.md index 9e6b5e6d281..f9ca3919c27 100755 --- a/docs/book-summaries/readme.md +++ b/docs/book-summaries/readme.md @@ -86,6 +86,7 @@ - [Power and Prediction](book-summaries/power-and-prediction.md) - [7 Powers: The Foundations of Business Strategy](book-summaries/7-powers-the-foundations-of-business-strategy.md) - [Others](book-summaries/others.md) +- [Books / Blinkist - Productivity](book-summaries/books-blinkist-productivity.md) - [Books / Blinkist - Management](../management/books-blinkist-management) - [Product Book Summaries](management/product-management/product-book-summaries.md) - [Books To Read](books-to-read) diff --git a/docs/knowledge/others/news-problems.md b/docs/knowledge/others/news-problems.md index 63d2820af09..377c6030a79 100644 --- a/docs/knowledge/others/news-problems.md +++ b/docs/knowledge/others/news-problems.md @@ -12,6 +12,10 @@ [Watch Out!! This Is What The CENTRAL BANKS Are Planning! - YouTube](https://www.youtube.com/watch?v=wdpQoO8af6g) +## World News + +- Pakistan - [SAMAA TV - YouTube](https://www.youtube.com/channel/UCJekW1Vj5fCVEGdye_mBN6Q) + ## Links [thoughts-future-ideas-vision-prediction](about-me/ideas/thoughts-future-ideas-vision-prediction.md) diff --git a/docs/media/Pasted image 20240917123040.png b/docs/media/Pasted image 20240917123040.png new file mode 100644 index 00000000000..f9fced981d9 Binary files /dev/null and b/docs/media/Pasted image 20240917123040.png differ diff --git a/docs/media/Pasted image 20240917123109.png b/docs/media/Pasted image 20240917123109.png new file mode 100644 index 00000000000..e399786c0fc Binary files /dev/null and b/docs/media/Pasted image 20240917123109.png differ diff --git a/docs/media/Screenshot 2024-09-17 at 1.46.23 AM.jpg b/docs/media/Screenshot 2024-09-17 at 1.46.23 AM.jpg new file mode 100644 index 00000000000..cf2a6f5bb18 Binary files /dev/null and b/docs/media/Screenshot 2024-09-17 at 1.46.23 AM.jpg differ diff --git a/docs/psychology/productivity-productive-time-management-todo.md b/docs/psychology/productivity-productive-time-management-todo.md index 98385e35b16..313e4ab9769 100755 --- a/docs/psychology/productivity-productive-time-management-todo.md +++ b/docs/psychology/productivity-productive-time-management-todo.md @@ -85,13 +85,7 @@ A Most Important Task (MIT) is a critical task that will create the most signifi - [Parkinson's Law](https://en.wikipedia.org/wiki/Parkinson%27s_law) - "Work expands so as to fill the time available for its completion." - (3) Gate's Law - "Most people overestimate what they can do in one year and underestimate what they can do in ten years." -## Blinkist - The 12 Week Year by Brian P. Moran - -Get more done in 12 weeks than others do in 12 months - -Plan around a 12-week period, execute tasks and radically improve results - -### Choosing a productivity system +## Choosing a productivity system 1. Your system needs to fit your work (not the other way around) 2. The system should counterbalance your worst tendencies. @@ -134,74 +128,6 @@ At first this seems crazy large. And, admittedly, this rule can be reduced for l https://www.scotthyoung.com/blog/2020/12/07/planning -## Blinkist - Get your shit together by Sarah Knight (2 Jan 2019) - -![image](../media/Productivity-Productive-Time-Management-TODO-image2.jpg) - -## Blinkist - 5 Gears: How to Be Present and Productive When There is Never Enough Time by Jeremie Kubicek, Steve Cockram (7 Jan 2019) - -- **First gear -** when you fully rest and recharge (Learning to recharge) -- **Second gear -** when you connect with family or friends without the involvement of work (Connecting deeply) -- **Third gear -** when you are socializing (Why being social matters) -- **Fourth gear -** when you are working and multitasking (Leading in the task world) -- **Fifth gear -** when you are fully focused and 'in the zone,' working without interruption (In the Zone) -- **Reverse -** Being responsive in a resistant world - -## Blinkist - Great at Work by Morten T. Hansen - -How top performers do less, work better, and achieve more - -- Mastering your own work - - Do less, then obsess - - Choosing a few priorities - - Dedicating your efforts toward excelling at them - - Redesign your work - - Start with a value not with a goal - - Don't just learn, loop - - Micro-behavior is a small, concrete action you take on a daily basis to improve a skill. The action shouldn't take more than fifteen minutes to perform and review, and it should have a clear impact on skill development. - - P-squared (Passion and Purpose) -- Mastering working with others - - Forceful champions - - Fight and unite - - Fight - On a meeting, maximize a team debate - - Unite - Commit to decision at the end of meeting, because best idea wins - - The two sins of collaboration -- Master your work-life - -## 168 Hours By Laura Vanderkam - -You Have More Time Than You Think - -- You're not too busy -- you're just not in control of your time. - - Instead of thinking, "I'm too busy," you should ask, "Is that a priority?" If it isn't, simply let it go. -- To live a fulfilling life, focus on your core competencies. -- Choosing a job that uses your core competencies energizes your entire life. - - "Any 'work' that is not advancing you towards the professional life you want should not count as work. It is wasted time." -- To become time-rich you must control your work calendar. - - Create habit of excusing yourself from meetings. If you realize a meeting isn't relevant to you, and it's a waste of your time, you should leave. And that time would be better spent pursuing activities that advance your career. -- To achieve a career breakthrough, combine strategy with story. - - "There's a lot of randomness in the universe, but truly lucky people recognize that fairy godmothers are lazy." -- You can improve your relationships with your children by focusing on your core competencies. - - If you both love swimming, schedule regular trips to the local pool. A love of music might mean singing together. -- Outsource housework that doesn't align with your core competencies. -- Plan your leisure time to increase its value. -- Actionable advice: Win family time by creating a mini night shift. - - Working parents often find it difficult to spend time with their children during the week. Overcome this by blocking out 5:00 p.m. to 8:00 p.m. in your calendar every weeknight -- and view this as sacred family time. Counter your shorter day at the office by working from 8:00 p.m. to 10:00 p.m. a few nights a week. By doing this, you'll maximize your time together while the kids are still awake. Otherwise, you're more likely to waste your evening leisure time in front of the TV. - -## The Lazy Genius Way by Kendra Adachi - -Embrace what matters, ditch what doesn't, and get stuff done - -- What's in it for me? Learn how to focus on the things that actually matter -- Being a perfectionist is exhausting and lonely -- but there is another way -- Real transformation happens in small steps -- Making fixed decisions can bring calm and focus -- You can defuse stressful situations with a little planning -- Introducing a few simple rules can transform your home -- In order to discover what's important to you, you'll need to take some time out -- We can't escape difficult situations, but we can change how we respond to them -- We need to become our own best friends -- and then allow other people in - ## Chunking 1. Start by capturing @@ -231,22 +157,18 @@ Liner is a browser extension that lets you highlight anything you see online. Be ## Resources -https://www.freecodecamp.org/news/how-to-get-things-done-lessons-in-productivity - -https://deepstash.com/article/1700/23-time-management-techniques-of-insanely-busy-people - -https://www.youtube.com/watch?v=fBr8BKPW5tc - -https://medium.com/swlh/the-5-most-productive-things-to-do-when-youre-too-tired-to-work-4157efe9091b - -https://hellonehha.hashnode.dev/how-to-protect-your-time-at-work-cksoxjhtq079dvks125jhewf3 - -https://www.productivitygame.com - -- [Personal Productivity](https://www.youtube.com/watch?v=-f3dNkkFBvM) +- https://www.freecodecamp.org/news/how-to-get-things-done-lessons-in-productivity +- https://deepstash.com/article/1700/23-time-management-techniques-of-insanely-busy-people +- https://www.youtube.com/watch?v=fBr8BKPW5tc +- https://medium.com/swlh/the-5-most-productive-things-to-do-when-youre-too-tired-to-work-4157efe9091b +- https://hellonehha.hashnode.dev/how-to-protect-your-time-at-work-cksoxjhtq079dvks125jhewf3 +- https://www.productivitygame.com + - [Personal Productivity](https://www.youtube.com/watch?v=-f3dNkkFBvM) +- [Productivity as a Developer – Full Course - YouTube](https://www.youtube.com/watch?v=C_jQahOnGUU) ## Links +- [Books / Blinkist - Productivity](book-summaries/books-blinkist-productivity.md) - [Organize Tomorrow Today](../book-summaries/organize-tomorrow-today) - [The Slight Edge](../book-summaries/the-slight-edge) - [The 4 Hour Workweek](../book-summaries/the-4-hour-workweek)