0.7.0

Highlights

• We restructured and updated our documentation, which should make it easier to get started, and learn more advanced concepts as you go.

  👉 Check it out at fondant.ai!

• We moved our example pipelines into separate repositories, which will make it easier to get started with them:

  • 📖 RAG ingestion pipeline
    An end-to-end Fondant pipeline that prepares documents for a RAG (Retrieval Augmented Generation) system by chunking and embedding them, and writing them to a vector store.

  • 🛋️ ControlNet Interior Design Pipeline
    An end-to-end Fondant pipeline to collect and process data for the fine-tuning of a ControlNet model, focusing on images related to interior design.

  • 🖼️ Filter creative common license images
    An end-to-end Fondant pipeline that starts from our Fondant-CC-25M creative commons image dataset and filters and downloads the desired images.

  • 🔢 Datacomp pipeline
    An end-to-end Fondant pipeline filtering image-text data to train a CLIP model for the DataComp competition.

• We split our component and pipeline SDK, so only the actual required dependencies are installed.

  In components, install the component extra so you can use the fondant.component SDK.

  pip install fondant[component]

  Locally, just install Fondant without extras to fondant.pipeline SDK and CLI.

  For the local runner:

  pip install fondant

  Or with the appropriate extra for your specific runner:

  pip install fondant[vertex]

  More info on the available installation options is available here.

All changes

• bugfix typo in text_normalization by @andres-vv in #542
• Fondant build - add test case and documentation by @mrchtr in #546
• Add search bar to the documentation by @mrchtr in #545
• Fix cli error propagation by @PhilippeMoussalli in #544
• Remove Starcoder pipeline by @PhilippeMoussalli in #552
• Simplify cloud credentials mounting by @PhilippeMoussalli in #548
• Bump gcsfs by @PhilippeMoussalli in #553
• Remove starcoder pipeline reference from README.md by @mrchtr in #556
• Set mkdocs site_url to fix 404 page by @RobbeSneyders in #559
• Add screenshots data explorer by @mrchtr in #555
• Split component and pipeline SDKs by @RobbeSneyders in #587
• Datacomp submission improvement by @PhilippeMoussalli in #586
• Update readme generator reference to new sdk by @PhilippeMoussalli in #591
• Add componentOp warning for unused configuration by @PhilippeMoussalli in #551
• Add component install extra and update others by @RobbeSneyders in #592
• Fixing pre-commit building component READMEs by @mrchtr in #596
• Granular compiler tests by @PhilippeMoussalli in #589
• Fix pipeline label by @PhilippeMoussalli in #606
• Fix naming error by @PhilippeMoussalli in #607
• Remove example pipelines by @mrchtr in #588
• Add simple pipeline and integration test for the LocalRunner by @mrchtr in #594
• Delete datacomp pipeline by @mrchtr in #612
• Restructure documentation by @PhilippeMoussalli in #597
• Add more info about caching by @PhilippeMoussalli in #615
• Update langchain version by @mrchtr in #617
• Update hub by @PhilippeMoussalli in #609
• Add Content Tab to document python and Console SDK by @PhilippeMoussalli in #613
• Docs site improvements by @RobbeSneyders in #620
• Expand docker installation guide by @PhilippeMoussalli in #619
• publishing Components by @PhilippeMoussalli in #621
• Update README by @RobbeSneyders in #622
• Use names instead of directory names in hub by @RobbeSneyders in #626
• Add label argument to fondant build by @mrchtr in #623
• Update name by @PhilippeMoussalli in #630
• Remove faq by @PhilippeMoussalli in #629
• Small formatting fixes by @PhilippeMoussalli in #614
• Add architecture description and plot by @PhilippeMoussalli in #628
• Skip followed imports for referencing.jsonschema by @RobbeSneyders in #644
• Add authentification arg to run command by @PhilippeMoussalli in #645
• Small doc fixes by @PhilippeMoussalli in #648
• Remove explicit fsspec requirements from data explorer by @RobbeSneyders in #650

New Contributors

• @andres-vv made their first contribution in #542

Full Changelog: 0.6.2...0.7.0