updated docs

docs/about-me/ideas/education.md

@@ -217,6 +217,12 @@ Save 100+ hours in prelims note making. Notes based on psychological method of '
 
 Just add a module of UPSC in khan academy. It has everything.
 
+## Interview Prep Model
+
+- Do mock interviews
+- Tell what's wrong with the candidate
+- Share the module / courses and full plan on how to be better
+
 ## Links
 
 - [He's helped THOUSANDS of people learn to code \[Leon Noel of #100devs on freeCodeCamp Podcast #117\] - YouTube](https://www.youtube.com/watch?v=KYAakLooVuE&ab_channel=freeCodeCamp.org)

docs/about-me/ideas/startups-ideas-saas.md

@@ -304,3 +304,4 @@ Lawyer / law - central knowledge base and marketplace
 
 - [The Grid](../../book-summaries/the-grid)
 - [Top 100 Startup Ideas from Top 100 Entrepreneurs/VCs/Hackers! - YouTube](https://www.youtube.com/watch?v=XiXLti_Y_is&ab_channel=Breakdown)
+- [9 startups that stood out on YC Demo Day 2 | TechCrunch](https://techcrunch.com/2024/09/26/9-startups-that-stood-out-on-yc-demo-day-2/)

docs/about-me/projects/95-zenatix-solutions.md

@@ -1 +0,0 @@
-

docs/about-me/projects/96-zenatix-architecture.md

@@ -1 +0,0 @@
-

docs/ai/deep-learning/resources.md

@@ -15,6 +15,7 @@
 
 ## Courses
 
+- [5 Free Courses to Master Deep Learning in 2024 - MachineLearningMastery.com](https://machinelearningmastery.com/5-free-courses-to-master-deep-learning-in-2024/)
 - Paid - https://www.coursera.org/specializations/deep-learning
 - Paid - [Deep Learning Online Training Course | Udacity](https://www.udacity.com/course/deep-learning-nanodegree--nd101)
 - PAID - Coursera - https://www.deeplearning.ai

docs/ai/llm/libraries.md

@@ -123,6 +123,8 @@
 - [**maccarone**](https://github.com/bsilverthorn/maccarone) - AI-managed code blocks in Python, lets you delegate sections of your Python program to AI ownership.
 - [**magentic**](https://github.com/jackmpcollins/magentic) - prompt LLMs as simple Python functions using decorators.
 - [**Semantic Kernel**](https://github.com/microsoft/semantic-kernel) - integrate cutting-edge LLM technology quickly and easily into your apps. Microsoft’s “version” of [LangChain](https://github.com/hwchase17/langchain).
+- [ControlFlow](https://controlflow.ai/welcome)
+	- ControlFlow is a Python framework for building agentic AI workflows.
 
 ### LLM Code Tools
 
@@ -151,15 +153,6 @@
 - [**Langflow**](https://github.com/logspace-ai/langflow) - UI for [LangChain](https://github.com/hwchase17/langchain), designed with react-flow to provide an effortless way to experiment and prototype flows.
 - [**PromptTools**](https://github.com/hegelai/prompttools) - a set of open-source, self-hostable tools for experimenting with, testing, and evaluating LLMs, vector databases, and prompts. [HN launch post](https://news.ycombinator.com/item?id=36958175).
 
-### LLM Metrics / Evaluation
-
-- [**DeepEval**](https://github.com/confident-ai/deepeval) - a simple-to-use, open-source evaluation framework for LLM applications.
-- [**Fiddler Auditor**](https://github.com/fiddler-labs/fiddler-auditor) - a tool to evaluate the robustness of language models.
-- [**ragas**](https://github.com/explodinggradients/ragas) - Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines.
-- [**tvalmetrics**](https://github.com/TonicAI/tvalmetrics) - Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.
-
-[A Gentle Introduction to LLM Evaluations - Elena Samuylova - YouTube](https://www.youtube.com/live/ac6ZB5QEwGU)
-
 ### LLM Serving
 
 - [**Aviary**](https://github.com/ray-project/aviary) - an LLM serving solution that makes it easy to deploy and manage a variety of open source LLMs. By the authors of [Ray](https://github.com/ray-project/ray).

docs/ai/llm/models.md

@@ -107,6 +107,15 @@ Emotional prompting example - You are Dolphin, an uncensored and unbiased Al ass
 - [Hugging Face Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
 - [Alpaca Eval Leaderboard](https://tatsu-lab.github.io/alpaca_eval/)
 - [GitHub - tatsu-lab/alpaca\_eval: An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.](https://github.com/tatsu-lab/alpaca_eval)
+- [A Gentle Introduction to LLM Evaluations - Elena Samuylova - YouTube](https://www.youtube.com/live/ac6ZB5QEwGU)
+- [Eureka: OSS Framework to evaluate LLMs - by Bugra Akyildiz](https://mlops.substack.com/p/eureka-oss-framework-to-evaluate)
+
+### Tools
+
+- [**DeepEval**](https://github.com/confident-ai/deepeval) - a simple-to-use, open-source evaluation framework for LLM applications.
+- [**Fiddler Auditor**](https://github.com/fiddler-labs/fiddler-auditor) - a tool to evaluate the robustness of language models.
+- [**ragas**](https://github.com/explodinggradients/ragas) - Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines.
+- [**tvalmetrics**](https://github.com/TonicAI/tvalmetrics) - Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.
 
 ## SAAS Models
 

docs/ai/llm/tools.md

@@ -19,6 +19,7 @@
 - [ChatGPT Code Interpreter is like a Data Scientist on steroids](https://twitter.com/moritzkremb/status/1654107314528612355)
 - [The Amazing AI Super Tutor for Students and Teachers | Sal Khan | TED - YouTube](https://www.youtube.com/watch?v=hJP5GqnTrNo&ab_channel=TED)
 - [WebChatGPT: ChatGPT with internet access | Chrome Web Store - Extensions](https://chrome.google.com/webstore/detail/webchatgpt-chatgpt-with-i/lpfemeioodjbpieminkklglpmhlngfcn/related)
+- [NotebookLM | Note Taking & Research Assistant Powered by AI](https://notebooklm.google/)
 
 ## AI Generators
 

docs/ai/ml-fundamentals/readme.md

@@ -2,6 +2,7 @@
 
 - [Table of Contents](table-of-contents)
 - [Intro](ai/ml-fundamentals/intro.md)
+- [Loss Function](ai/ml-fundamentals/loss-function.md)
 - [ML Classification](ml-classification)
 - [ML Process](ml-process)
 - [Preliminaries](preliminaries)

docs/cloud/aws/compute/aws-lambda.md

@@ -83,15 +83,15 @@ In asynchronous invocation, the caller doesn’t wait for the function’s respo
 
 ### 2 - Assignment Service
 
-The Assignment Service manages the execution environments. 
+The Assignment Service manages the execution environments.
 
-The service is written in Rust for high performance and is divided into multiple partitions with a leader-follower approach for high availability. 
+The service is written in Rust for high performance and is divided into multiple partitions with a leader-follower approach for high availability.
 
 The state of execution environments is written to an external journal log.
 
 ### 3 - Firecracker MicroVM
 
-Firecracker is a lightweight virtual machine manager designed for running serverless workloads such as AWS Lambda and AWS Fargate. 
+Firecracker is a lightweight virtual machine manager designed for running serverless workloads such as AWS Lambda and AWS Fargate.
 
 It uses Linux’s Kernel-based virtual machine to create and manage secure, fast-booting microVMs.
 
@@ -103,4 +103,4 @@ To make it efficient, it uses multiple techniques:
 
 - Chunking to store the container images more efficiently.
 - Using convergent encryption to secure the shared data. This involves appending additional data to the chunk to compute a more robust hash.
-- SnapStart feature to reduce cold start latency by pre-initializing the execution environment 
+- SnapStart feature to reduce cold start latency by pre-initializing the execution environment

docs/cloud/aws/compute/ec2-instance-types.md

@@ -231,3 +231,8 @@ https://aws.amazon.com/ec2/instance-types/g4
 
 - [Upgrade EC2 instance | AWS re:Post](https://repost.aws/questions/QUu8o1AEwmSgyZJho1hEfuRw/upgrade-ec2-instance)
 - [Resolve boot errors on Linux Nitro-based EC2 instances | AWS re:Post](https://repost.aws/knowledge-center/boot-error-linux-nitro-instance)
+
+## Links
+
+- [Amazon EC2 R8g Instances](https://aws.amazon.com/ec2/instance-types/r8g/)
+	- R8g instances offer up to 30% better performance and larger instance sizes with up to 3x more vCPUs and memory than the seventh-generation AWS Graviton3-based R7g instances.

docs/cloud/aws/security-identity-compliance/compliance.md

@@ -163,6 +163,7 @@ The role of compliance officers is crucial for any company to ensure that the bu
 ## Others
 
 - [Data Governance](ai/data-science/data-governance.md)
+- [GitHub - rkm-ravi94/awesome-it-compliance: This repo contains some details about the IT compliances available.](https://github.com/rkm-ravi94/awesome-it-compliance)
 - CISA Certification - Certified Information Systems Auditor
 - CISO - Chief Information Security Officer
 - CMMI Level 3 - An appraisal at maturity level 3 **indicates an organization is performing at a “defined” level**. At this level, processes are well characterized and understood and are described in standards, procedures, tools, and methods.

docs/computer-science/interview-question/system-design-youtube.md

@@ -11,4 +11,3 @@
 7. The Transcoding Status Handler updates the metadata DB and cache with the latest details of the video.
 8. The user raises a video streaming request that goes to a Content Delivery Network (CDN).
 9. The CDN fetches the video from the object storage for streaming. It also caches the video locally for subsequent streaming requests.
-

docs/computer-science/operating-system/caching-strategies-topologies.md

@@ -1,38 +1,38 @@
 # Caching Strategies
 
-1. Cooperative/Distributed caching
+### 1. Cooperative/Distributed caching
 
-    Cache data is distributed between nodes
+Cache data is distributed between nodes
 
-    In Cooperative Caching, also known as Distributed Caching, multiple distinct systems (normally referred to as cluster-nodes) work together to build a huge, shared cache.
+In Cooperative Caching, also known as Distributed Caching, multiple distinct systems (normally referred to as cluster-nodes) work together to build a huge, shared cache.
 
-2. Partial caching
+### 2. Partial caching
 
-    Partial Caching describes a type of caching where not all data is stored inside the cache. Depending on certain criteria, responses might not be cacheable or are not expected to be cached (like temporary failures).
+Partial Caching describes a type of caching where not all data is stored inside the cache. Depending on certain criteria, responses might not be cacheable or are not expected to be cached (like temporary failures).
 
-    A typical example for data where not everything is cacheable is websites. Some pages are "static" and
+A typical example for data where not everything is cacheable is websites. Some pages are "static" and
 
-    only change if some manual or regular action happens. Those pages can easily be cached and invalidated whenever this particular action happened. Apart from that, other pages consist of mostly dynamic content or frequently updated content (like stock market tickers) and shouldn't be cached at all.
+only change if some manual or regular action happens. Those pages can easily be cached and invalidated whenever this particular action happened. Apart from that, other pages consist of mostly dynamic content or frequently updated content (like stock market tickers) and shouldn't be cached at all.
 
-3. Geographical caching
+### 3. Geographical caching
 
-    Geographical Caches are located in strategically chosen locations to optimize latency on requests, therefore this kind of cache will mostly be used for website content. It is also known as CDN (Content Delivery Network).
+Geographical Caches are located in strategically chosen locations to optimize latency on requests, therefore this kind of cache will mostly be used for website content. It is also known as CDN (Content Delivery Network).
 
-4. Preemptive caching
+### 4. Preemptive caching
 
-    A Preemptive Cache itself is not a caching type like the others above but is mostly used in conjunction with a Geographical Cache.
+A Preemptive Cache itself is not a caching type like the others above but is mostly used in conjunction with a Geographical Cache.
 
-    Using a warm-up engine a Preemptive Cache is populated on startup and tries to update itself based on rules or events. The idea behind this cache addition is to reload data from any backend service or central cluster even before a requestor wants to retrieve the element. This keeps access time to the cached elements constant and prevents accesses to single elements from becoming unexpectedly long.
+Using a warm-up engine a Preemptive Cache is populated on startup and tries to update itself based on rules or events. The idea behind this cache addition is to reload data from any backend service or central cluster even before a requestor wants to retrieve the element. This keeps access time to the cached elements constant and prevents accesses to single elements from becoming unexpectedly long.
 
-    Building a Preemptive Cache is not easy and requires a lot of knowledge of the cached domain and the update workflows.
+Building a Preemptive Cache is not easy and requires a lot of knowledge of the cached domain and the update workflows.
 
-5. Latency SLA caching
+### 5. Latency SLA caching
 
-    A Latency SLA Cache is able to maintain latency SLAs even if the cache is slow or overloaded. This type of cache can be build in two different ways.
+A Latency SLA Cache is able to maintain latency SLAs even if the cache is slow or overloaded. This type of cache can be build in two different ways.
 
-    The first option is to have a timeout to exceed before the system either requests the potentially cached element from the original source (in parallel to the already running cache request) or provides a simple default answer, and uses whatever returns first.
+The first option is to have a timeout to exceed before the system either requests the potentially cached element from the original source (in parallel to the already running cache request) or provides a simple default answer, and uses whatever returns first.
 
-    The other option is to always fire both requests in parallel and take whatever returns first. This option is not the preferred way of implementation since it mostly works against the idea of caching and won't reduce load on the backend system. This option might still make sense if multiple caching layers are available, like always try the first and second nearest caches in parallel.
+The other option is to always fire both requests in parallel and take whatever returns first. This option is not the preferred way of implementation since it mostly works against the idea of caching and won't reduce load on the backend system. This option might still make sense if multiple caching layers are available, like always try the first and second nearest caches in parallel.
 
 ## Caching Topologies
 
@@ -129,3 +129,29 @@ There are multiple layers along the flow.
 - Replication Log: used to record the replication state in a database cluster
 
 ![image](../../media/cache-layers-data.jpg)
+
+## Integrated Redis Cache
+
+![Integrated redis cache](../../media/Pasted%20image%2020241006232918.jpg)
+
+### 1. CacheFront Read and Writes with CDC
+
+- Uber built CacheFront - an integrated caching solution with Redis, Docstore, and MySQL.
+
+- Rather than the microservice, Docstore’s query engine communicates with Redis for read requests.
+- For cache hits, the query engine fetches data from Redis. For cache misses, the request goes to the storage engine and the database.
+- In the case of writes, Docstore’s CDC service (Flux) invalidates the records in Redis. It tails MySQL binlog events to trigger the invalidation.
+
+### 2. Multi-Region Cache Warming with Redis Streaming
+
+- A region fail-over can result in cache misses and overload the database.
+- To handle this, Uber’s engineering team uses cross-region Redis replication. This is done by tailing the Redis write stream to replicate keys to the remote region.
+- In the remote region, the stream consumer issues read requests to the query engine that reads the database and updates the cache.
+
+### 3. Redis and Docstore Sharding
+
+- All teams in Uber use Docstore and some generate a huge number of requests.
+- Both Redis and Docstore instances are sharded or partitioned to handle the load. But a single Redis cluster going down may create a hot DB shard.
+- To prevent this, they partitioned the Redis cluster using a scheme that was different from the DB sharding. This ensures that the load is evenly distributed.
+
+[EP131: How Uber Served 40 Million Reads with Integrated Redis Cache?](https://blog.bytebytego.com/p/ep131-how-uber-served-40-million)

docs/computer-science/system-design/architecture-guide.md

@@ -26,7 +26,7 @@ Web-queue-worker is suitable for relatively simple domains with some resource-in
 | Big data                  | Divide a huge dataset into small chunks. Parallel processing on local datasets.  | Batch and real-time data analysis. Predictive analysis using ML. |
 | Big compute               | Data allocation to thousands of cores.                                           | Compute intensive domains such as simulation.                    |
 
-### - Design Principles
+### Design Principles
 
 - Design for self-healing
 - Make all things redundant
@@ -39,7 +39,7 @@ Web-queue-worker is suitable for relatively simple domains with some resource-in
 - Design for evolution
 - Build for the needs of business
 
-### - Best Practices
+### Best Practices
 
 - API Design
 - API Implementation
@@ -54,13 +54,13 @@ Web-queue-worker is suitable for relatively simple domains with some resource-in
 - Retry Guidance for Specific services
 - Transient fault handling
 
-### - Performance Tuning
+### Performance Tuning
 
 - Scenario 1 - Distributed Transactions
 - Scenario 2 - Multiple backend services
 - Scenario 3 - Event Streaming
 
-### - Performance Antipatterns
+### Performance Antipatterns
 
 - Busy Database
 - Busy Front End

docs/computer-science/system-design/intro.md

@@ -211,5 +211,6 @@ This helps as you no longer have to make assumptions about
 - [System Design Tutorial: 3 Must-Know Distributed Systems Concepts | by Arslan Ahmad | Geek Culture | Medium](https://medium.com/geekculture/system-design-tutorial-3-must-know-distributed-systems-concepts-279d4e9718e8)
 - [Brief History of Scaling Uber](https://highscalability.com/brief-history-of-scaling-uber/)
 - [Behind AWS S3’s Massive Scale](https://highscalability.com/behind-aws-s3s-massive-scale/)
+- [Software Architecture Patterns - ByteByteGo Newsletter](https://blog.bytebytego.com/p/software-architecture-patterns)
 
 ![System Design Cheatsheet](../../media/Pasted%20image%2020240607132531.png)

docs/computer-science/system-design/others.md

@@ -116,3 +116,18 @@ You can do this by implementing or configuring it in the Play Store or App Store
 A good place to store these versions is in your cache (Redis/Memcache), which you can change on the fly without needing to deploy your application.
 
 [Stop using client-side route redirects](https://kentcdodds.com/blog/stop-using-client-side-route-redirects)
+
+## Distributed Locks
+
+A distributed lock is a mechanism that ensures mutual exclusion across a distributed system.
+
+### Top 6 Use Cases for Distributed Locks
+
+1. **Leader Election -** Distributed locks can be used to ensure that only one node becomes the leader at any given time.
+2. **Task Scheduling -** In a distributed task scheduler, distributed locks ensure that a scheduled task is executed by only one worker node, preventing duplicate execution.
+3. **Resource Allocation -** When managing shared resources like file systems, network sockets, or hardware devices, distributed locks ensure that only one process can access the resource at a time.
+4. **Microservices Coordination -** When multiple microservices need to perform coordinated operations, such as updating related data in different databases, distributed locks ensure that these operations are performed in a controlled and orderly manner.
+5. **Inventory Management -** In e-commerce platforms, distributed locks can manage inventory updates to ensure that stock levels are accurately maintained when multiple users attempt to purchase the same item simultaneously.
+6. **Session Management -** When handling user sessions in a distributed environment, distributed locks can ensure that a user session is only modified by one server at a time, preventing inconsistencies.
+
+![Distributed lock use cases](../../media/Pasted%20image%2020240924221049.jpg)

docs/databases/nosql-databases/cassandra/intro.md

@@ -101,3 +101,4 @@ In the image above, circles are Cassandra nodes and lines between the circles sh
 - https://www.freecodecamp.org/news/the-apache-cassandra-beginner-tutorial
 - [Introduction to Apache Cassandra™ + What's New in 4.0 by Patrick McFadin | DataStax Presents](https://www.youtube.com/watch?v=d7o6a75sfY0)
 - [Cassandra Tutorial Videos](https://www.youtube.com/playlist?list=PL9ooVrP1hQOGJ4Yz9vbytkRmLaD6weg8k)
+- [How Uber Scaled Cassandra for Tens of Millions of Queries Per Second?](https://blog.bytebytego.com/p/how-uber-scaled-cassandra-for-tens)