Skip to content

Commit

Permalink
updated docs
Browse files Browse the repository at this point in the history
  • Loading branch information
deepaksood619 committed Jan 9, 2024
1 parent 9ffa2a3 commit f049b85
Show file tree
Hide file tree
Showing 11 changed files with 106 additions and 34 deletions.
3 changes: 3 additions & 0 deletions docs/ai/data-science/data-visualization/bi-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,4 +75,7 @@ https://aws.amazon.com/blogs/big-data/add-comparative-and-cumulative-date-time-c
https://www.talend.com/

[GitHub - finos/perspective: A data visualization and analytics component, especially well-suited for large and/or streaming datasets.](https://github.com/finos/perspective)

[ThoughtSpot | The AI-Powered Analytics Platform](https://www.thoughtspot.com/)

[Reporting and embedded business intelligence software | Jaspersoft](https://www.jaspersoft.com/)
5 changes: 5 additions & 0 deletions docs/databases/indexing/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,8 @@
- [Database Index](database-index)
- [Inverted Index](inverted-index)
- [MySQL Indexing](mysql-indexing)

## Others

- [Postgres Indexes](databases/sql-databases/postgres/indexes.md)
- [Mongodb Indexes](databases/nosql-databases/mongodb/indexes.md)
2 changes: 1 addition & 1 deletion docs/databases/nosql-databases/mongodb/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

- [MongoDB Intro](databases/nosql-databases/mongodb/intro.md)
- [Overview](overview)
- [Indexes](indexes)
- [Indexes](databases/nosql-databases/mongodb/indexes.md)
- [Data Types](databases/nosql-databases/mongodb/data-types.md)
- [Commands](databases/nosql-databases/mongodb/commands.md)
- [pymongo](pymongo)
Expand Down
72 changes: 72 additions & 0 deletions docs/databases/sql-databases/postgres/indexes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Indexes in Postgres

[PostgreSQL: Documentation: 16: Chapter 11. Indexes](https://www.postgresql.org/docs/current/indexes.html)

## Types

### [B-tree indexes](https://www.postgresql.org/docs/current/btree-intro.html)

B-tree indexes are binary trees that are used to sort data efficiently. They're the default if you use the INDEX command. Most of the time, a B-tree index suffices. As you scale, inconsistencies can be a larger problem, so use the [amcheck](https://www.postgresql.org/docs/11/amcheck.html) extension periodically.

### Hash Indexes

Hash indexes store a 32-bit hash code derived from the value of the indexed column. Hence, such indexes can only handle simple equality comparisons. The query planner will consider using a hash index whenever an indexed column is involved in a comparison using the equal operator

[PostgreSQL: Documentation: 16: 11.2. Index Types](https://www.postgresql.org/docs/current/indexes-types.html#INDEXES-TYPES-HASH)

### [BRIN indexes](https://www.postgresql.org/docs/11/brin-intro.html)

A Block Range INdex (BRIN) can be used when your table is naturally already sorted by a column, and you need to sort by that column. For example, for a log table that was written sequentially, setting a BRIN index on the timestamp column lets the server know that the data is already sorted.

BRIN works in terms of _block ranges_ (or “page ranges”). A block range is a group of pages that are physically adjacent in the table; for each block range, some summary info is stored by the index. For example, a table storing a store's sale orders might have a date column on which each order was placed, and most of the time the entries for earlier orders will appear earlier in the table as well; a table storing a ZIP code column might have all codes for a city grouped together naturally.

[PostgreSQL: Documentation: 16: 71.1. Introduction](https://www.postgresql.org/docs/current/brin-intro.html)

[Postgres BRIN Index — Large Data Performance With Minimal Storage | by Eresh Gorantla | Geek Culture | Medium](https://medium.com/geekculture/postgres-brin-index-large-data-performance-with-minimal-storage-4db6b9f64ca4)

[Block Range Index - Wikipedia](https://en.wikipedia.org/wiki/Block_Range_Index)

### [Bloom filter index](https://www.postgresql.org/docs/11/bloom.html)

A bloom index is perfect for multi-column queries on big tables where you only need to test for equality. It uses a special mathematical structure called a bloom filter that's based on probability and uses significantly less space.

### [GIN and GiST indexes](https://www.postgresql.org/docs/11/textsearch-indexes.html)

Use a GIN or GiST index for efficient indexes based on composite values like text, arrays, and JSON.

There are two kinds of indexes that can be used to speed up full text searches. Note that indexes are not mandatory for full text searching, but in cases where a column is searched on a regular basis, an index is usually desirable.

```sql
-- Creates a GiST (Generalized Search Tree)-based index. The column can be of tsvector or tsquery type.
CREATE INDEX name ON table USING gist(column);

-- Creates a GIN (Generalized Inverted Index)-based index. The column must be of tsvector type.
CREATE INDEX name ON table USING gin(column);
```

A GiST index is _lossy_, meaning that the index may produce false matches, and it is necessary to check the actual table row to eliminate such false matches.

GIN indexes are not lossy for standard queries, but their performance depends logarithmically on the number of unique words.

Performance
- GIN index lookups are about three times faster than GiST
- GIN indexes take about three times longer to build than GiST
- GIN indexes are moderately slower to update than GiST indexes, but about 10 times slower if fast-update support was disabled.
- GIN indexes are two-to-three times larger than GiST indexes

https://habr.com/en/company/postgrespro/blog/448746

[PostgreSQL: Documentation: 9.1: GiST and GIN Index Types](https://www.postgresql.org/docs/9.1/textsearch-indexes.html)
### SP-GiST

SP-GiST indexes, like GiST indexes, offer an infrastructure that supports various kinds of searches. SP-GiST permits implementation of a wide range of different non-balanced disk-based data structures, such as quadtrees, k-d trees, and radix trees (tries).

[PostgreSQL: Documentation: 16: 11.2. Index Types](https://www.postgresql.org/docs/current/indexes-types.html#INDEXES-TYPE-SPGIST)

## Partial Indexes

_partial index_ is an index built over a subset of a table; the subset is defined by a conditional expression (called the _predicate_ of the partial index). The index contains entries only for those table rows that satisfy the predicate.

[PostgreSQL: Documentation: 16: 11.8. Partial Indexes](https://www.postgresql.org/docs/current/indexes-partial.html)

[PostgreSQL - Partial Index - GeeksforGeeks](https://www.geeksforgeeks.org/postgresql-partial-index/)
30 changes: 10 additions & 20 deletions docs/databases/sql-databases/postgres/readme.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Postgres

- [Postgres Documentation](postgres/documentation)
- [Table Partitioning](databases/sql-databases/postgres/table-partitioning.md)
- [Postgres Indexes](databases/sql-databases/postgres/indexes.md)
- [parameters-configuration-optimization](databases/sql-databases/postgres/parameters-configuration-optimization.md)
- [others](databases/sql-databases/postgres/others.md)

## Introduction

Is a relational database management system with an object-oriented approach, meaning that information can be represented as objects or classes in PostgreSQL schemas.
Expand Down Expand Up @@ -79,26 +85,6 @@ https://severalnines.com/database-blog/overview-json-capabilities-within-postgre

Containment tests whether one document (a set or an array) is contained within another.

## Indexes in Postgres

#### [B-tree indexes](https://www.postgresql.org/docs/current/btree-intro.html)

B-tree indexes are binary trees that are used to sort data efficiently. They're the default if you use theINDEXcommand. Most of the time, a B-tree index suffices. As you scale, inconsistencies can be a larger problem, so use the [amcheck](https://www.postgresql.org/docs/11/amcheck.html) extension periodically.

#### [BRIN indexes](https://www.postgresql.org/docs/11/brin-intro.html)

A Block Range INdex (BRIN) can be used when your table is naturally already sorted by a column, and you need to sort by that column. For example, for a log table that was written sequentially, setting a BRIN index on the timestamp column lets the server know that the data is already sorted.

#### [Bloom filter index](https://www.postgresql.org/docs/11/bloom.html)

A bloom index is perfect for multi-column queries on big tables where you only need to test for equality. It uses a special mathematical structure called a bloom filter that's based on probability and uses significantly less space.

#### [GIN and GiST indexes](https://www.postgresql.org/docs/11/textsearch-indexes.html)

Use a GIN or GiST index for efficient indexes based on composite values like text, arrays, and JSON.

https://habr.com/en/company/postgrespro/blog/448746

## Streaming replication asynchronous and synchronous

https://severalnines.com/database-blog/converting-asynchronous-synchronous-replication-postgresql
Expand Down Expand Up @@ -131,6 +117,10 @@ Youtube - [Breaking PostgreSQL at Scale - Christophe Pettus](https://www.youtube

Tools - pgadmin

## SAAS

- [EDB: Open-Source, Enterprise Postgres Database Management](https://www.enterprisedb.com/)

## References

- http://www.postgresqltutorial.com
Expand Down
4 changes: 0 additions & 4 deletions docs/databases/sql-databases/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,5 @@
- [AWS Redshift](aws-redshift/readme.md)
- [MySQL](mysql/readme.md)
- [Postgres](postgres/readme.md)
- [Postgres Documentation](postgres/documentation)
- [Table Partitioning](databases/sql-databases/postgres/table-partitioning.md)
- [parameters-configuration-optimization](databases/sql-databases/postgres/parameters-configuration-optimization.md)
- [others](databases/sql-databases/postgres/others.md)
- [memsql](memsql/readme.md)
- [Intro](memsql/intro)
2 changes: 2 additions & 0 deletions docs/knowledge/quotes-proverbs/business-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -453,3 +453,5 @@ Working ON means taking a zoomed out approach and thinking: what systems could w
If your business can't run without you - if you can't safely leave your business for a few weeks - you don't really have a business; you have a job.

Tobacco lobby releasing a study on the health benefits of smoking

![client-meme](../../media/Screenshot_20240106-223754.png)
Binary file added docs/media/Pasted image 20240109123958.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/Screenshot_20240106-223754.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 6 additions & 9 deletions docs/psychology/parenting.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,12 +95,9 @@ Schooling isn't just about education, it is also a form a childcare

## Links

[Colin Powell: Kids need structure | TED - YouTube](https://www.youtube.com/watch?v=NhYnouvrG_8&ab_channel=TED)

[Indian Parents , Their Pravachan And An Important Parenting Tip - YouTube](https://www.youtube.com/watch?v=Sjttt-F2pXE)

[The Single Most Important Parenting Strategy - Repair | Becky Kennedy | TED - YouTube](https://www.youtube.com/watch?v=PHpPtdk9rco)

[Toddlers Fears](https://www.whattoexpect.com/toddler-development/toddler-fears.aspx)

[How to talk to babies](https://www.whattoexpect.com/first-year/milestones/how-to-talk-to-babies)
- [Colin Powell: Kids need structure | TED - YouTube](https://www.youtube.com/watch?v=NhYnouvrG_8&ab_channel=TED)
- [Indian Parents , Their Pravachan And An Important Parenting Tip - YouTube](https://www.youtube.com/watch?v=Sjttt-F2pXE)
- [The Single Most Important Parenting Strategy - Repair | Becky Kennedy | TED - YouTube](https://www.youtube.com/watch?v=PHpPtdk9rco)
- [Toddlers Fears](https://www.whattoexpect.com/toddler-development/toddler-fears.aspx)
- [How to talk to babies](https://www.whattoexpect.com/first-year/milestones/how-to-talk-to-babies)
- [7 Signs that Witnessing Unhealthy Anger in Childhood Hurts You Today - YouTube](https://www.youtube.com/watch?v=yyTfxRUu8gc)
7 changes: 7 additions & 0 deletions docs/technologies/apache/airflow/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,3 +101,10 @@ https://www.youtube.com/playlist?list=PLCi-q9vYo4x-PESoBcXN0tXCMgzh5c_Pj
## Debugging

https://www.astronomer.io/blog/7-common-errors-to-check-when-debugging-airflow-dag

## Others

- Astronomer
- Amazon Managed Apache Airflow [What Is Amazon Managed Workflows for Apache Airflow? - Amazon Managed Workflows for Apache Airflow](https://docs.aws.amazon.com/mwaa/latest/userguide/what-is-mwaa.html)

![Amazon MWAA Architecture](../../../media/Pasted%20image%2020240109123958.png)

0 comments on commit f049b85

Please sign in to comment.