Skip to content

Commit

Permalink
add Safetensors (#447)
Browse files Browse the repository at this point in the history 
* add Safetensors

* Update README.md

* Update README.md

* Update README.md
  • Loading branch information
@zhimin-z
zhimin-z authored Jan 1, 2024
1 parent 69b59f4 commit f6d6818
Showing 1 changed file with 9 additions and 8 deletions.
17 changes: 9 additions & 8 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ This repository contains a curated list of awesome open source libraries that wi
| [🧵 Data Pipeline](#data-pipeline) | [🏷️ Data Labelling & Synthesis](#data-labelling-and-synthesis) | [📅 Metadata Management](#metadata-management) |
| [🗺️ Computation Distribution](#computation-load-distribution) | [📥 Model Serialisation](#model-serialisation) | [🧮 Optimized Computation](#optimized-computation)|
| [💸 Data Stream Processing](#data-stream-processing) | [:red_circle: Outlier & Anomaly Detection](#outlier-and-anomaly-detection) | [🎁 Feature Store](#feature-store) |
| [⚔ Adversarial Robustness](#adversarial-robustness) | [💾 Data Storage Optimization](#data-storage-optimisation) | [📓 Data Science Notebook](#data-science-notebook) |
| [⚔ Adversarial Robustness](#adversarial-robustness) | [💾 Data Storage Optimisation](#data-storage-optimisation) | [📓 Data Science Notebook](#data-science-notebook) |
| [🔥 Neural Search](#neural-search) | [👁️ Industry-strength Computer Vision](#industry-strength-cv) | [🔠 Industry-strength Natural Language Processing](#industry-strength-nlp) |
| [🍕 Industry-strength Reinforcement Learning](#industry-strength-rl) | [📊 Industry-strength Visualisation](#industry-strength-visualisation) | [🙌 Industry-strength Recommender System](#industry-strength-recsys) |
| [📈 Industry-strength Benchmarking & Evaluation](#industry-strength-benchmarking-and-evaluation) | [💰 Commercial Platform](#commercial-platform) |
Expand Down Expand Up @@ -72,7 +72,7 @@ This repository contains a curated list of awesome open source libraries that wi
* [CleverHans](https://github.com/tensorflow/cleverhans) ![](https://img.shields.io/github/stars/tensorflow/cleverhans.svg?style=social) - An adversarial example library for constructing attacks, building defenses, and benchmarking both. A python library to benchmark system's vulnerability to [adversarial examples](http://karpathy.github.io/2015/03/30/breaking-convnets/).
* [ContrastiveExplanation (Foil Trees)](https://github.com/MarcelRobeer/ContrastiveExplanation) ![](https://img.shields.io/github/stars/MarcelRobeer/ContrastiveExplanation.svg?style=social) - Python script for model agnostic contrastive/counterfactual explanations for machine learning. Accompanying code for the paper ["Contrastive Explanations with Local Foil Trees"](https://arxiv.org/abs/1806.07470).
* [DeepLIFT](https://github.com/kundajelab/deeplift) ![](https://img.shields.io/github/stars/kundajelab/deeplift.svg?style=social) - Codebase that contains the methods in the paper ["Learning important features through propagating activation differences"](https://arxiv.org/abs/1704.02685). Here is the [slides](https://docs.google.com/file/d/0B15F_QN41VQXSXRFMzgtS01UOU0/edit?filetype=mspresentation) and the [video](https://vimeo.com/238275076) of the 15 minute talk given at ICML.
* [DeepVis Toolbox](https://github.com/yosinski/deep-visualization-toolbox) ![](https://img.shields.io/github/stars/yosinski/deep-visualization-toolbox.svg?style=social) - This is the code required to run the Deep Visualization Toolbox, as well as to generate the neuron-by-neuron visualizations using regularized optimization. The toolbox and methods are described casually [here](http://yosinski.com/deepvis) and more formally in this [paper](https://arxiv.org/abs/1506.06579).
* [DeepVis Toolbox](https://github.com/yosinski/deep-visualization-toolbox) ![](https://img.shields.io/github/stars/yosinski/deep-visualization-toolbox.svg?style=social) - This is the code required to run the Deep Visualization Toolbox, as well as to generate the neuron-by-neuron visualizations using regularized optimisation. The toolbox and methods are described casually [here](http://yosinski.com/deepvis) and more formally in this [paper](https://arxiv.org/abs/1506.06579).
* [ELI5](https://github.com/TeamHG-Memex/eli5) ![](https://img.shields.io/github/stars/TeamHG-Memex/eli5.svg?style=social) - "Explain Like I'm 5" is a Python package which helps to debug machine learning classifiers and explain their predictions.
* [FACETS](https://github.com/PAIR-code/facets) ![](https://img.shields.io/github/stars/PAIR-code/facets.svg?style=social) - Facets contains two robust visualizations to aid in understanding and analyzing machine learning datasets. Get a sense of the shape of each feature of your dataset using Facets Overview, or explore individual observations using Facets Dive.
* [Fairlearn](https://github.com/fairlearn/fairlearn) ![](https://img.shields.io/github/stars/fairlearn/fairlearn.svg?style=social) - Fairlearn is a python toolkit to assess and mitigate unfairness in machine learning models.
Expand Down Expand Up @@ -252,12 +252,12 @@ This repository contains a curated list of awesome open source libraries that wi
* [FLAML](https://github.com/microsoft/FLAML) ![](https://img.shields.io/github/stars/microsoft/FLAML.svg?style=social) - FLAML is a fast library for automated machine learning & tuning.
* [go-featureprocessing](https://github.com/nikolaydubina/go-featureprocessing) ![](https://img.shields.io/github/stars/nikolaydubina/go-featureprocessing.svg?style=social) - A feature pre-processing framework in Go that matches functionality of sklearn.
* [Katib](https://github.com/kubeflow/katib) ![](https://img.shields.io/github/stars/kubeflow/katib.svg?style=social) - A Kubernetes-based system for Hyperparameter Tuning and Neural Architecture Search.
* [keras-tuner](https://github.com/keras-team/keras-tuner) ![](https://img.shields.io/github/stars/keras-team/keras-tuner?style=social) - Keras Tuner is an easy-to-use, distributable hyperparameter optimization framework that solves the pain points of performing a hyperparameter search. Keras Tuner makes it easy to define a search space and leverage included algorithms to find the best hyperparameter values.
* [keras-tuner](https://github.com/keras-team/keras-tuner) ![](https://img.shields.io/github/stars/keras-team/keras-tuner?style=social) - Keras Tuner is an easy-to-use, distributable hyperparameter optimisation framework that solves the pain points of performing a hyperparameter search. Keras Tuner makes it easy to define a search space and leverage included algorithms to find the best hyperparameter values.
* [Maggy](https://github.com/logicalclocks/maggy) ![](https://img.shields.io/github/stars/logicalclocks/maggy.svg?style=social) - Asynchronous, directed Hyperparameter search and parallel ablation studies on Apache Spark - [(Video)](https://www.youtube.com/watch?v=0Hd1iYEL03w).
* [Neural Architecture Search with Controller RNN](https://github.com/titu1994/neural-architecture-search) ![](https://img.shields.io/github/stars/titu1994/neural-architecture-search.svg?style=social) - Basic implementation of Controller RNN from [Neural Architecture Search with Reinforcement Learning](https://arxiv.org/abs/1611.01578) and [Learning Transferable Architectures for Scalable Image Recognition](https://arxiv.org/abs/1707.07012).
* [Neural Network Intelligence](https://github.com/Microsoft/nni) ![](https://img.shields.io/github/stars/Microsoft/nni.svg?style=social) - NNI (Neural Network Intelligence) is a toolkit to help users run automated machine learning (AutoML) experiments.
* [Optuna](https://github.com/optuna/optuna) ![](https://img.shields.io/github/stars/optuna/optuna.svg?style=social) - Optuna is an automatic hyperparameter optimization software framework, particularly designed for machine learning.
* [OSS Vizier](https://github.com/google/vizier) ![](https://img.shields.io/github/stars/google/vizier.svg?style=social) - OSS Vizier is a Python-based service for black-box optimization and research, one of the first hyperparameter tuning services designed to work at scale.
* [Optuna](https://github.com/optuna/optuna) ![](https://img.shields.io/github/stars/optuna/optuna.svg?style=social) - Optuna is an automatic hyperparameter optimisation software framework, particularly designed for machine learning.
* [OSS Vizier](https://github.com/google/vizier) ![](https://img.shields.io/github/stars/google/vizier.svg?style=social) - OSS Vizier is a Python-based service for black-box optimisation and research, one of the first hyperparameter tuning services designed to work at scale.
* [sklearn-deap](https://github.com/rsteca/sklearn-deap) ![](https://img.shields.io/github/stars/rsteca/sklearn-deap.svg?style=social) Use evolutionary algorithms instead of gridsearch in scikit-learn.
* [TPOT](https://github.com/epistasislab/tpot) ![](https://img.shields.io/github/stars/epistasislab/tpot.svg?style=social) - Automation of sklearn pipeline creation (including feature selection, pre-processor, etc.).
* [tsfresh](https://github.com/blue-yonder/tsfresh) ![](https://img.shields.io/github/stars/blue-yonder/tsfresh.svg?style=social) - Automatic extraction of relevant features from time series.
Expand Down Expand Up @@ -357,6 +357,7 @@ This repository contains a curated list of awesome open source libraries that wi
* [Marqo](https://github.com/marqo-ai/marqo) ![](https://img.shields.io/github/stars/marqo-ai/marqo.svg?style=social) Marqo is an end-to-end vector search engine.
* [pgvector](https://github.com/pgvector/pgvector) ![](https://img.shields.io/github/stars/pgvector/pgvector.svg?style=social) pgvector helps with vector similarity search for Postgres.
* [PostgresML](https://github.com/postgresml/postgresml) ![](https://img.shields.io/github/stars/postgresml/postgresml.svg?style=social) PostgresML is a machine learning extension for PostgreSQL that enables you to perform training and inference on text and tabular data using SQL queries.
* [Safetensors](https://github.com/huggingface/safetensors) ![](https://img.shields.io/github/stars/huggingface/safetensors.svg?style=social) Simple, safe way to store and distribute tensors.
* [TimescaleDB](https://github.com/timescale/timescaledb) ![](https://img.shields.io/github/stars/timescale/timescaledb.svg?style=social) An open-source time-series SQL database optimized for fast ingest and complex queries packaged as a PostgreSQL extension - [(Video)](www.youtube.com/watch?v=zbjub8BQPyE).
* [Weaviate](https://github.com/semi-technologies/weaviate) ![](https://img.shields.io/github/stars/semi-technologies/weaviate.svg?style=social) - A low-latency vector search engine (GraphQL, RESTful) with out-of-the-box support for different media types. Modules include Semantic Search, Q&A, Classification, Customizable Models (PyTorch/TensorFlow/Keras), and more.
* [Zarr](https://github.com/zarr-developers/zarr-python) ![](https://img.shields.io/github/stars/zarr-developers/zarr-python.svg?style=social) - Python implementation of chunked, compressed, N-dimensional arrays designed for use in parallel computing.
Expand Down Expand Up @@ -413,8 +414,8 @@ This repository contains a curated list of awesome open source libraries that wi
* [Jax](https://github.com/google/jax) ![](https://img.shields.io/github/stars/google/jax.svg?style=social) - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more.
* [MLX](https://github.com/ml-explore/mlx) ![](https://img.shields.io/github/stars/ml-explore/mlx.svg?style=social) - MLX is an array framework for machine learning on Apple silicon.
* [Modin](https://github.com/modin-project/modin) ![](https://img.shields.io/github/stars/modin-project/modin.svg?style=social) - Speed up your Pandas workflows by changing a single line of code.
* [Nebullvm](https://github.com/nebuly-ai/nebullvm) ![](https://img.shields.io/github/stars/nebuly-ai/nebullvm.svg?style=social) - Nebullvm is an ecosystem of plug and play modules to boost the performances of your AI systems. The optimization modules are stack-agnostic and work with any library. They are designed to be easily integrated into your system, providing a quick and seamless boost to its performance.
* [Nevergrad](https://github.com/facebookresearch/nevergrad) ![](https://img.shields.io/github/stars/facebookresearch/nevergrad.svg?style=social) - Nevergrad is a gradient-free optimization platform.
* [Nebullvm](https://github.com/nebuly-ai/nebullvm) ![](https://img.shields.io/github/stars/nebuly-ai/nebullvm.svg?style=social) - Nebullvm is an ecosystem of plug and play modules to boost the performances of your AI systems. The optimisation modules are stack-agnostic and work with any library. They are designed to be easily integrated into your system, providing a quick and seamless boost to its performance.
* [Nevergrad](https://github.com/facebookresearch/nevergrad) ![](https://img.shields.io/github/stars/facebookresearch/nevergrad.svg?style=social) - Nevergrad is a gradient-free optimisation platform.
* [Norse](https://github.com/norse/norse) ![](https://img.shields.io/github/stars/norse/norse.svg?style=social) - Norse aims to exploit the advantages of bio-inspired neural components, which are sparse and event-driven - a fundamental difference from artificial neural networks.
* [Numba](https://github.com/numba/numba) ![](https://img.shields.io/github/stars/numba/numba.svg?style=social) - A compiler for Python array and numerical functions.
* [NumpyGroupies](https://github.com/ml31415/numpy-groupies) ![](https://img.shields.io/github/stars/ml31415/numpy-groupies.svg?style=social) Optimised tools for group-indexing operations: aggregated sum and more
Expand Down Expand Up @@ -704,7 +705,7 @@ This repository contains a curated list of awesome open source libraries that wi
* [SambaNova](https://sambanova.ai/) - SambaNova Systems is a company that specializes in generative AI. They offer a full-stack platform that allows users to build powerful AI models, customized with their data, and owned by them.
* [Scale](https://scale.com) - Scale AI turns raw data into high-quality training data by combining machine learning powered pre-labeling and active tooling with varying levels and types of human review.
* [Scribble Enrich](https://www.scribbledata.io/product) - Customizable, auditable, privacy-aware feature store. It is designed to help mid-sized data teams gain trust in the data that they use for training and analysis, and support emerging needs such drift computation and bias assessment.
* [SigOpt](https://sigopt.com/) - SigOpt is a model development platform that makes it easy to track runs, visualize training, and scale hyperparameter optimization for any type of model built with any library on any infrastructure.
* [SigOpt](https://sigopt.com/) - SigOpt is a model development platform that makes it easy to track runs, visualize training, and scale hyperparameter optimisation for any type of model built with any library on any infrastructure.
* [Skymind](https://skymind.global/) - Software distribution designed to help enterprise IT teams manage, deploy, and retrain machine learning models at scale.
* [Skytree](http://skytree.net) - End to end machine learning platform - [(Video)](https://www.youtube.com/watch?v=XuCwpnU-F1k).
* [SuperAnnotate](https://www.superannotate.com/) - A complete set of solutions for image and video annotation and an annotation service with integrated tooling, on-demand narrow expertise in various fields, and a custom neural network, automation, and training models powered by AI.
Expand Down

0 comments on commit f6d6818

Please sign in to comment.