Skip to content

Latest commit

 

History

History
1037 lines (919 loc) · 162 KB

CHANGELOG.md

File metadata and controls

1037 lines (919 loc) · 162 KB
Raw

Changelog

All notable changes to this project will be documented in this file.

0.5.3 - 2024-09-04

⛰️ Features

🐛 Bug Fixes

  • Fix bug in redis store when zero data stored but data does not exist (#1304) - (59020f1)
  • Fix bug where OperationId::String was being used instead of Uuid version (#1301) - (cc611cd)
  • Fix rare case where eof was sent on buf_channel when retry happens (#1295) - (47dfc20)
  • Fix Tekton depedency order within Pulumi (#1291) - (0fd0a94)
  • Revert "Release NativeLink v0.5.2 (#1283)" (#1284) - (1b38a64)

📚 Documentation

  • Re-enable docs auto-deployment on main (#1317) - (ca88d90)
  • Migrate docs buildsystem from pnpm to bun (#1268) - (ef3a8a6)

🧪 Testing & CI

  • Fix nix2container skopeo patch hash (#1294) - (689d099)
  • Fix broken variables in NativeLink Cloud CI jobs and disable RBE test (#1293) - (f4ae4cc)

⚙️ Miscellaneous

  • S3 store will now retry more aggresively (#1302) - (0ecf5b4)
  • Remove nix2container patch hash workaround (#1296) - (d5c55ac)
  • Use docker to create a buck2 image (#1275) - (8896b65)
  • Support remote build execution on main and read-only remote cache on PRs (#1277) - (2f9fd8b)
  • Release NativeLink v0.5.2 (#1283) - (a528481)

⬆️ Bumps & Version Updates

0.5.2 - 2024-08-23

🐛 Bug Fixes

  • Fix verify_size w/ verify_hash set to true in VerifyStore (#1273) - (c21d59f)

📚 Documentation

🧪 Testing & CI

⚙️ Miscellaneous

  • Revert "Make de/serialized structs compliant with Rust naming practices (#1271)" (#1282) - (0933c1a)
  • Make de/serialized structs compliant with Rust naming practices (#1271) - (a174fbf)
  • Append buck2 toolchain with additional packages (#1264) - (042f4a5)
  • Remove ActionScheduler and introduce KnownPlatformPropertyProvider (#1260) - (9c87370)
  • add static size and fix meta-typo (#1261) - (bddee33)
  • Raise correct error if BEP service fails (#1259) - (6b7401a)
  • Crosscompile NativeLink (#1233) - (ab64efd)

⬆️ Bumps & Version Updates

0.5.1 - 2024-08-08

🐛 Bug Fixes

  • [Bug] Add rt-tokio feature to aws-sdk-s3 (#1248) - (3eadab0)

⚙️ Miscellaneous

  • Release NativeLink v0.5.1 (#1252) - (b961192)
  • Conversion implementations for awaited action db structs (#1243) - (d5f2781)
  • Make redis clients available on RedisStore (#1244) - (c3f648e)

0.5.0 - 2024-08-07

❌️ Breaking Changes

  • [Breaking] Digest function now auto-detected from request (#899) - (0a33c83)

⛰️ Features

  • Add example clang/rust/go toolchain (#1200) - (11298d8)
  • Introduce NL_LOG to control logging format (#1154) - (d9922b3)
  • Add Capacitor dashboard to devcluster (#1115) - (93ae95a)
  • Add Flux to development cluster (#1096) - (6a40374)
  • Allow Tekton pipelines to be triggered by Flux Alerts (#1094) - (5de75cc)
  • Allow WebSocket upgrades in devcluster Loadbalancer (#1098) - (dda8c31)
  • Implement RedisStateManager (#1023) - (5104778)
  • Add optional and experimental pub sub publisher for redis store write. (#1027) - (128ba2a)
  • Decouple nativelink from toolchain containers (#1013) - (00e5bb3)
  • Add Bazel rules for generating rust-project.json (#1019) - (bb91fa9)
  • Add list api to StoreApi and MemoryStore (#1003) - (5a78919)
  • Add memory store optimized subscription API (#988) - (bf9edc9)
  • Add serialize and deserialize to structs (#965) - (79908cb)
  • Add subscribe API to Store API (#924) - (3be7255)
  • Add a config option to prefix keys in Redis stores (#981) - (b7a7e36)
  • Add OrderBy field for OperationFilter (#969) - (a911af4)
  • Add initial support for BEP (Build Event Protocol) (#961) - (23cba13)
  • Convert RedisError into nativelink Error (#959) - (cabc0c3)
  • Add JSON config examples to store.rs (#967) - (da9399b)
  • Make quantity field human readable (#891) - (da2c4a7)
  • Add drake toolchain configs (#942) - (e65c04a)
  • Add Operation State Manager API (#937) - (1d2d838)

🐛 Bug Fixes

📚 Documentation

🧪 Testing & CI

  • Remove some needless CI tests (#1240) - (3e259fd)
  • Fix Cargo.toml files when using cargo test on specific packages (#1236) - (ba7abf3)
  • Remove nativelink-proto as build dependency (#1209) - (19f4483)
  • Significantly reduce Bazel test time (#1210) - (4f49d53)
  • [Refactor] Overhaul of scheduler component (#1169) - (3b8c3a5)
  • Add BEP to CI (#1124) - (fa7b099)
  • Fix bystream_server_tests (#1087) - (846b25b)
  • Reduce references to self.state_manager.inner (#1060) - (2eefa75)
  • Fixes cyclical dependency between util and store (#1017) - (200f976)
  • [bug] Ensure OperationId is used at external protocol points (#1001) - (5ffaf89)
  • Remove installation test from devShell (#1014) - (9c40d57)
  • Increase timeout of pre-commit-checks CI pipeline (#1009) - (2d64361)
  • Add CI test to run on nativelink.com (#1007) - (3bc14bd)
  • Create scheduler state module (#968) - (264edb7)
  • Remove extraneous mod statements from tests (#975) - (f59a1d7)
  • Add dev build profile and remove lto from CI (#976) - (cec25fb)
  • Fix pulumi ratelimiting build error (#953) - (03841cc)
  • Add kind-loadbalancer (#929) - (c42fd0d)

⚙️ Miscellaneous

  • Release NativeLink v0.5.0 (#1223) - (d898c54)
  • Migrate much of the ActionScheduler API to ClientStateManager API (#1241) - (2b8f1ee)
  • Move ActionSchedulerListener to ActionStateResult (#1237) - (d57ee8d)
  • modified the lre file path (#1239) - (33f09cb)
  • Remove ClientOperationId and move all to OperationId (#1214) - (81db90e)
  • Remove unnecessary sync trait bounds. (#1227) - (e26e1b5)
  • Migrate from redis-rs to fred.rs (#1188) - (44a4a91)
  • Convert AwaitedAction to and from raw bytes (#1206) - (f004351)
  • Make Cargo.toml feature pins compatible with project/main (#1212) - (d8c407a)
  • Remove unused features in dependencies (#1211) - (a501971)
  • ExistenceCacheStore now only evicts based on insert (#1203) - (250037f)
  • Remove unused dependencies (#1207) - (df5f9e2)
  • Migrate to hyper 1.x, axum 0.7.x, tonic 0.12.x (#1155) - (532d1b1)
  • S3 store can ignore .has() requests based on LastModified (#1205) - (e874baa)
  • [Refactor] Complete metrics overhaul (#1192) - (a6ff968)
  • Migrate to callPackage syntax (#1193) - (534a102)
  • Implement Serialize/Deserialize for ActionStage (#1186) - (3574149)
  • update store_trait.rs (#1184) - (97f64b2)
  • Double protect output stream of verify store (#1180) - (e6542e6)
  • Make TaskExecutor a wrapper around TokioExecutor (#1159) - (b7ef3b6)
  • Increase chromium deployment example jobs size (#1146) - (0e265dc)
  • Refresh readme (#1078) - (414289a)
  • Change remote cache URLs from secrets to vars (#1143) - (6e37f47)
  • converted single defaults from plural (#1099) - (0a05082)
  • Write Tekton image tag outputs to a ConfigMap (#1100) - (1b8e23b)
  • Temporarily disable rustdoc autogen (#1101) - (3aa4f94)
  • Cancel running GHA workflows on pushes to the same branch (#1090) - (545f752)
  • Make bystream limits configurable (#1076) - (54a9345)
  • [Refactor] Workers::find_worker_for_action should take PlatformProperties (#1068) - (f5e7276)
  • Include ActionState to MatchingEngineActionStateResult (#1064) - (35e9cd7)
  • revert bazel version bump. (#1061) - (194ab78)
  • Remove #[async_trait] where possible (#620) (#1055) - (ba168a3)
  • Rename cas CompressionAlgorithm to HttpCompressionAlgorithm (#1052) - (9ba4323)
  • Implement MatchingEngineStateManager (#1041) - (684dbc1)
  • Move update_action_with_internal_error into StateManager (#1053) - (0f33a8a)
  • Implement WorkerStateManager for simple scheduler (#993) - (1359513)
  • Remove execution permissions from non-executable files (#1048) - (fbc39f5)
  • Sync serde version in Cargo.toml to lockfile (#966) - (59df55d)
  • Support cluster mode when using Redis as a store (#998) - (c85b6df)
  • Implement ClientStateManager for SimpleScheduler (#985) - (49efde2)
  • Reduce native-cli executable size (#1010) - (d1a8d9d)
  • Sync Cargo MSRV to Bazel (#1011) - (c0b284d)
  • [Refactor] Stores now return Arc for construction (#989) - (5bdc9eb)
  • Enable the dotcom workflow on main (#1008) - (28314e4)
  • EvictingMap now supports B-tree lookups (#996) - (fd4c89c)
  • [refactor] Migrate worker::WorkerId for action_messages::WorkerId (#992) - (50401c3)
  • [Refactor] Simple scheduler method signatures to async (#971) - (3c50dd5)
  • Refactor Store API to use StoreKey (#964) - (e524bbc)
  • Refactor Store Api into client side and driver side (#935) - (04beafd)
  • Create New Glossary (#957) - (77b2c33)
  • Use single quotes for char (#955) - (e90c4bc)
  • Include UUID in ActionState (#927) - (b07ca1d)
  • Refactor EvictingMap so it does not use DigestInfo (#932) - (9c45e86)

⬆️ Bumps & Version Updates

0.4.0 - 2024-05-16

❌️ Breaking Changes

  • [Breaking] Factor out health status checks to its own service (#823) - (ea50856)

⛰️ Features

🐛 Bug Fixes

  • Fix possible deadlock if max_open_files set too low (#908) - (e0a7bb9)
  • Fix LLVM 18 toolchains after fb0edae (#883) - (8ee7ab3)
  • Migrate K8s HTTPRoutes to GRPCRoutes (#868) - (7e379ff)
  • Fix bug in buf_channel::consume() where exact size doesn't receive eof (#858) - (5583a5d)

📚 Documentation

🧪 Testing & CI

  • Test building with Nix (#920) - (3391fdf)
  • Harden CI against too long running jobs (#917) - (ba7ed50)
  • Fix operations scripts evaluating to quickly (#906) - (66a72ab)
  • Add nativelink_test macro for tests (#888) - (c0d7eaa)

⚙️ Miscellaneous

  • Release NativeLink v0.4.0 (#913) - (75105df)
  • Reduce keep alive log message level (#894) - (f9e67aa)
  • Migrate to Bazelisk (#912) - (ab46197)
  • Enable hermetic Bazel sandboxing (#902) - (acec6d3)
  • All tokio::spawn and related functions must use nativelink's version (#890) - (c1d0402)
  • Remove zig-cc (#876) - (402f335)
  • Migrate all logging to the tracing library (#871) - (523ee33)
  • Refactor S3 store & support upload retry (#854) - (9db29ef)
  • fix a typo in the script comments. (#856) - (6d45a00)
  • Rename buf_channel::take() to buf_channel::consume() (#848) - (aadb2b9)
  • Connection Manager Rewrite (#806) - (a842f3a)

⬆️ Bumps & Version Updates

0.3.0 - 2024-04-10

❌️ Breaking Changes

  • [Breaking] Remove completeness checking logic in CacheLookupScheduler - (692e4de)
  • [Breaking] Generalize LRE to arbitrary toolchains (#728) - (1a43ef9)
  • [Breaking] Change in behavior of /status by introduction of component based health (#636) - (48cadc7)
  • [BREAKING] Add concurrency limit to GRPC (#627) - (b47f39b)
  • [Breaking] Deny unknown fields durning configuration serialization (#603) - (95afd36)

⛰️ Features

  • Add safe request timeout for running actions manager (#743) - (33db963)
  • Implement worker api for killing running actions (#840) - (abf12e8)
  • Create directory for action (#752) - (414fff3)
  • Add nativelink-debug target (#811) - (c60fb55)
  • Allow variables in platform property values (#809) - (09fc7f8)
  • Use mimalloc as global memory allocator (#749) - (6c647d6)
  • Optimize file uploads when source is file (#723) - (7c9a070)
  • Add API so stores can get Arc or &Store (#679) - (5df8a78)
  • Add check for slow store to be noop and conditionally replace with fast (#670) - (e402a10)
  • Max concurrent GrpcStore streams (#656) - (7548d4b)
  • Add metrics to compression and existence cache store (#651) - (722c80b)
  • Retry GrpcStore get_part_ref (#646) - (d46180c)
  • Allow ByteStream write restart (#635) - (3fabbaa)
  • Add warning for TLS (#609) - (63e2ad6)
  • Add support for mTLS (#470) - (6a379b3)
  • Add S3 http2 toggle flag (#604) - (8c433cd)
  • Add blake3 support for verify store (#575) - (3acefc7)
  • Build nativelink with musl (#583) - (ee4846c)
  • Shard store weight scale distribution (#574) - (928f12f)
  • Add console subscriber (#545) - (bb30474)

🐛 Bug Fixes

  • Fix semver image workflow after 646253d (#844) - (e890c01)
  • Resolve upload deadlock (#816) - (b61142d)
  • Fix nightly clippy warnings (#817) - (6d87cca)
  • Fix .gitignore after 1a43ef9 (#797) - (53e5a99)
  • Fix image publishing workflow after 1a43ef9 (#777) - (54b21b8)
  • Completeness checking store should not check if directory digests exist (#748) - (e979e31)
  • Check owner and group executable bits (#727) - (cea2336)
  • Fix case where resource_name not set in stream error (#746) - (a651f2c)
  • Set rust-version (#734) - (d2dd46d)
  • Account for block size in filesystem store for eviction purposes (#661) - (0639a59)
  • Fix cargo install tag and start command (#654) - (89313ff)
  • Don't retry permanent failures (#634) - (81b64f7)
  • Reenable caching for nix workflows (#631) - (6de799d)
  • Fix AMI NativeLink Tarballing (#645) - (c8473ac)
  • Evict on touch failure (#613) - (3037a66)
  • Disable flaky caching for LRE-Remote workflow (#619) - (2899f31)
  • Unbreak manual rustfmt invocations via Bazel (#617) - (f39e275)
  • Fix case where filesystem store future dropping causes issues (#496) - (249322d)
  • Minor refactor of functionally same code (#607) - (51715bd)
  • Fix a potential bug in DropCloserReadHalf::take() (#606) - (70e8525)
  • Fix dark mode accessibility contrast and made theme dynamic based on user machine (#597) - (d5443c8)

📚 Documentation

  • Update README.md to include License and Slack (#841) - (6c4fb7e)
  • Example of chromium using deployment scripts (#786) - (0aa7f65)
  • Update README for more clarity (#803) - (31a1bf1)
  • Fix incorrect bazel version 6.4.0+ in documenation (#801) - (b1b3bcb)
  • Update js dependencies in docs (#766) - (4b8eeaf)
  • Add search functionality to docs (#740) - (3dc1b8e)
  • Add configuration breakdown page (#725) - (35daf43)
  • Starts a Breakdown of Configuration (#680) - (433829c)
  • Draw a General Purpose Diagram (#705) - (2c102c3)
  • Basic config updated. (#669) - (f4d9db3)
  • Introduce Vale to lint documentation (#585) - (745b0d6)
  • Re-Add Rustup to the README (#648) - (0cba4fa)
  • Improve the LRE README (#637) - (63826f2)
  • Update README.md for AWS Terraform Deployment (#608) - (8a43fe4)
  • Add artifact warning to documentation and swap out cargo emoji (#599) - (89eafed)
  • Add Kubernetes Example to docs (#596) - (e1246fb)
  • Fix the bazel run command documentation (#590) - (7f4a007)
  • Add deployment examples to docs (#584) - (546484b)
  • Update README.md (#580) - (0269835)
  • Add OSFamily property in basic_cas.json (#577) - (3578d50)
  • Rearrange docs and aligned content with README (#571) - (beb87cf)

🧪 Testing & CI

⚙️ Miscellaneous

⬆️ Bumps & Version Updates

0.2.0 - 2023-12-21

❌️ Breaking Changes

  • [Breaking] Rename cas executable to nativelink (#573) - (ddf1d74)

📚 Documentation

🧪 Testing & CI

  • Add Nix formatters and linters to pre-commit hooks (#561) - (d823964)
  • Fix kill_all_waits_for_all_tasks_to_finish test stuck on windows (#525) - (143a5a1)
  • Fix missing timeouts in tests (#553) - (c54c51c)
  • Remove many of the large-* images in CI (#552) - (de0ae1e)

⚙️ Miscellaneous

⬆️ Bumps & Version Updates

  • Update Rust crate async-lock to v3 (#548) - (6c555bb)
  • Update OSSF domain (#558) - (82603d2)
  • Update LLVM and rust toolchains (#557) - (1726a1a)
  • Update actions/checkout action to v4 (#556) - (0d18d36)
  • Update Rust crate tokio to 1.35.1 (#535) - (c6f8b8a)
  • Update Rust crate tokio-rustls to 0.25.0 & rustls-pemfile to 2.0.0 (#540) - (cb76d18)
  • Update actions/checkout action to v3.6.0 (#541) - (5dce4ce)
  • Update dependency rules_python to v0.27.1 (#546) - (6ef8b6c)
  • Update dependency rules_rust to v0.34.1 (#547) - (637f283)
  • Update dependency @google-cloud/compute to v4.1.0 (#544) - (dbac23a)

[0.1.0] - 2023-12-20

❌️ Breaking Changes

  • [Breaking] Mark S3 store experimental - (05a6dd7)
  • [Breaking] listen_address renamed/remapped in config (#476) - (9db28d6)
  • [Breaking] Rename entrypoint_cmd->entrypoint and precondition_script (#475) - (dbe61d2)
  • [Breaking] Mark prometheus config as experimental (#473) - (931e721)
  • [Breaking] Standardize configurations so they are all lower case (#461) - (3329d7c)
  • [Breaking Change] Message field can now be populated (#361) - (cf2f3e4)
  • [Breaking Change] Add store type to GrpcStore. - (e1f3716)
  • [BreakingChange] Scheduler config now supports multiple impls - (384f14e)

⛰️ Features

  • Add renovate.json (#487) - (933963f)
  • Add OSFamily and container-image platform props (#512) - (b6b8252)
  • Add fancy badges (#521) - (e122042)
  • Add Git-Cliff Changelog (#515) - (8197bb9)
  • Integrate google analytics (#503) - (ef74f9c)
  • Add OpenSSF scorecard action (#486) - (4d9d897)
  • Add Completeness Checking Store (#404) - (d264624)
  • Publish container images (#443) - (697cddf)
  • Add function to Store API to get the inner store when possible (#410) - (a0788fa)
  • Add GCP to terraform deployment examples (#433) - (4661a36)
  • Add Blake3 digest support (#403) - (2c8f0f0)
  • Add Noop store (#408) - (aea3768)
  • Add DigestHasher as interface to hashing functions (#400) - (9e31ca4)
  • Add rustc explicitly to flake (#398) - (db724c0)
  • Add existence cache (#383) - (e8e6701)
  • Add ability for external scripts (ie: entrypoint_cmd) to manage timeout (#368) - (3ae120a)
  • Add Http2 flags for advanced configurations (#365) - (cb04ed4)
  • Add summary of platform properties to prometheus (#367) - (d9af3b9)
  • Add more err_tip for easier debugging (#363) - (b5ff95d)
  • Add security policy (#343) - (9173c2f)
  • Add retry to GrpcScheduler (#324) - (21519ce)
  • Add ability to ignore EOF check for writers (#341) - (979f941)
  • Introduce Nix development flake (#330) - (a0792fd)
  • Introduce Bazel build for Windows (#317) - (659d571)
  • Added tracking for all client connections since server started and time server started - (0375a8f)
  • Introduced shard store - (a7e3936)
  • Add Contributing file - (4900f06)
  • Add ADDITIONAL_SETUP_WORKER_CMD to Dockerfile - (3c30387)
  • Add windows support - (2875f0b)
  • Add support to build with Cargo - (bff3be3)
  • Add metrics to SimpleScheduler and Worker - (63f7393)
  • Add ability for metris to be disabled - (875b3ca)
  • Add property modifying scheduler. - (656e7f7)
  • Add metrics to LocalWorker and RunningActionsManager - (f0a526b)
  • Add prometheus stats to MemoryStore - (f274dcf)
  • Add retry to GrpcStore. - (259224b)
  • Add prometheus stats for VerifyStore - (5f5b2c4)
  • Add prometheus publishing and hook up FilesystemStore - (04a7772)
  • Add support for backpressure from workers. - (fc97fcb)
  • Add ability to create low watermark to avoid thrashing against eviction cap. - (e16b45c)
  • Add is_empty to LenEntry - (e643090)
  • Add timestamps to executor jobs. - (fa97b28)
  • Add support for environmental variable lookup in S3Store config - (cb0de9e)
  • Add ability to use env variables in config files - (d54b38e)
  • Add Send trait to as_any() store calls - (c4be423)
  • Add fs module which limits outstanding file handles - (f7b565f)
  • Add functionality for worker to download and create working dir - (5e7f9ef)
  • Adds .as_any() to stores - (e5de86d)
  • Adds initial implementation for LocalWorker and supporting classes - (90cff23)
  • Various minor updates - (cf6dd3d)
  • Add shlex package in third_party - (d935d7f)
  • Add worker config definitions and rename Metadata to Priority - (98c4e08)
  • Add WorkerApiServer to services being served - (af0ccc3)
  • Add support for keep alive for workers - (be6f2ee)
  • [RE] Add WorkerApiService and connection functionality - (e8a349c)
  • [RE] Various changes to worker_api.proto - (86220b7)
  • Add uuid package and update other packages - (5115bc6)
  • Add SizePartitioningStore - (d0112be)
  • Add RefStore and restructure StoreManager - (6795bb0)
  • Can now pass json config through CLI & add more sample configs - (ea4d76d)
  • Add nix package and upgrade third_party packages - (a451628)
  • Add basic scaffolding for scheduler + remote execution - (c91f61e)
  • Adds readme to configuration - (54e8fe7)
  • Add filesystem store - (d183cad)
  • Adds simple query_write_status support - (844014a)
  • Add buf_channel that will be used to help transport bytes around - (7e111c1)
  • Add byteorder to third_party cargo - (a76a35f)
  • Adds more eviction templates and functions in prep for filesystem store - (f2896a7)
  • Adds FastSlow store that will try the fast store before slow store - (8c71137)
  • Add dedup store - (2dba31c)
  • Add retry support to get_part in s3_store - (ea2fc4c)
  • Add CompressionStore and implement LZ4 compression - (d6cd4f9)
  • Add s3 configuration - (be87381)
  • Add retry utility in prep for s3_store - (86e63ee)
  • Add async_read_taker in prep for s3_store - (90222f9)
  • Add trust_size to DigestInfo - (d8f218f)
  • Add ability for VerifyStore to check the sha256 hash of the digest - (40ba2fb)
  • Add sha2 to Cargo.toml in prep for sha256 checking - (0eb2dab)
  • Add mock_instant library to Cargo.toml - (34b9312)
  • Add maplit to third_party dependencies - (b09153b)
  • Add json package dependencies and updates packages - (69cf723)
  • Add read stream support - (5c2db23)
  • Add drop_guard to Cargo.toml - (3c147cd)
  • Add ability to read partial store - (0b304cc)
  • Add multi-threading and fix some minor performance issues - (0ed309c)
  • Add DigestInfo utility - (25bef4a)
  • Add much better way to do error logging with .err_tip() - (9ae49b6)
  • Add futures package to Cargo.toml - (92912e6)
  • Add Capabilities and Execution api endpoints - (24dec02)
  • Add ./rust_fmt.sh - (5c65005)
  • Add dependent proto files for bazel cas - (d845d40)

🐛 Bug Fixes

  • Remove Fixed-Buffer Dependency (#509) - (5a6b182)
  • Fix rustfmt after 6d07a86 (#520) - (cfdf7e8)
  • Fixes error forwarding to client for failed command executions (#432) - (0c225da)
  • Fix unwrap function in the Prometheus server code (#446) - (406eab7)
  • Refactor filesystem store for timeout function passing (#439) - (5123ffc)
  • Handle SIGINT (#434) - (f9e537c)
  • Fixup configs to have defaults & digest function uses lower case (#438) - (d56f008)
  • Fix AWS terraform deployment (#423) - (4cc53bc)
  • Fix empty bytes error in s3 store and support AWS_ENDPOINT_URL (#421) - (cf531dc)
  • Migrate S3 store to official AWS SDK (#369) - (6ce11ab)
  • Fix double negative when computing remaining memory % in terraform deployment (#407) - (9e981a5)
  • Fix the typo of WorkerProperty (#391) - (8a1cb6b)
  • Retry GrpcStore write (#326) - (6006e23)
  • Revert "Fix never looping loops (#372)" (#373) - (8e234c5)
  • Fix never looping loops (#372) - (755c10e)
  • Close on complete in GrpcScheduler (#328) - (6c937da)
  • Fix potential race condition if worker disconnects - (b871a90)
  • Don't download zero size blobs - (c8e2ee8)
  • Fix prometheus metrics to not publish multiple times - (f42f150)
  • Fix readme TLDR - (b6a4046)
  • Fix default config regression in master - (bca2f3d)
  • Fix fence post bugs in dedup store - (d7c847c)
  • Fix the AWS deployment examples - (17bfbf6)
  • Fix inefficient upload of stderr/stdout in workers - (8ac4824)
  • Don't remove Error context. - (e9ab61e)
  • Fix clippy warnings for scheduler directory - (1491d0a)
  • Fix potential bug where scheduer could drop action - (f118ccd)
  • Fix "unused function" warnings in utf8_range - (f048352)
  • Fix digest clones and a few other minor clippy warnings - (a523115)
  • Fix clippy messages in cas/store - (7fef931)
  • Fix clippy erros for most other non-scheduler files - (264849b)
  • Fix clippy cas/grpc_service folder - (e85faed)
  • Fix most clippy warnings in worker files - (be228d0)
  • Fixes the entrypoint_cmd configuration - (096d7ea)
  • Fix a couple of nits with the timestamp additions. - (b320de5)
  • Fix bug if no instance_name/resource_name is given upload does not work - (b010b4b)
  • Fix scheduler so platform properties are properly restored - (059b0ef)
  • Fix bug on output_files' folders were not being created - (bb010f2)
  • Fix bug where worker was not creating working directory properly - (4e51b6d)
  • Fix wrong type_url in google-proto's Any type - (9cda96a)
  • Fix bug during .has() call in dedup store - (5cc9a09)
  • Fixed various bugs in filesystem store - (7ba407d)
  • Fix bug in evicting_map with unref improperly called and readability - (ea393a5)
  • Fix minor issue in FastSlowStore - (81fb378)
  • Fix case where s3 uploads in wrong order - (4798fe9)
  • Fix bug in s3_store where 5mb is calculated wrong & improve debugability - (0451781)
  • Fix s3_store - (efcb653)
  • Fixed AsyncFixedBuffer - (519fa9f)
  • Minor changes to AsyncFixedBuffer - (a506363)
  • Fix lifetime of StoreTrait::update() - (9ec43a2)
  • Fix --config debug config to properly add debug symbols - (90b43c6)
  • Fix small bug in gen_rs_proto - (627c0f8)
  • Fix small needless cast to i64 - (59c609e)
  • Fix bug with verify_store when receiving multiple chunks - (a78caec)
  • Fixed typo in debug message when instance_name is not properly set - (d231ea1)
  • Fixed EOF bits and few other items in order to get bazel working - (8558ee9)
  • Fix async_fixed_buffers to add get_closer() - (9225b1f)
  • Fix memory leak - (c27685c)
  • Fix Store import in cas_server.rs - (a7e7859)

📚 Documentation

  • Include command example for Powershell in documentation files (#501) - (0536d8e)
  • Add CodeQL scanning for Python and JS/TS (#484) - (34f0aa0)
  • Add documentation and machine type variables for gcp. (#457) - (cb6540c)
  • Rename docs directory (#468) - (43b4ea8)
  • Add docs to monorepo (#453) - (378b806)
  • Handle SIGTERM (#462) - (e49049c)
  • Make Native Link installable via nix (#442) - (b8f3ef1)
  • Adds README to docker-compose deployment-example (#427) - (3ec203b)
  • Fix the incorrect config path in the documentation (#416) - (7f40696)
  • Rewrite the build infrastructure (#394) - (3147265)
  • update the README for discoverability. (#349) - (5e2e81a)
  • Minor optimizations and documentation to CacheLookupScheduler - (66c403d)
  • Simplify Dockerfile and prepare for Goma example - (65b8f0e)
  • Update README.md - (7563df7)
  • Document that users should use -c opt for release builds - (9351f26)
  • Fix bazel version change that broke proto building and documentation - (1994dde)
  • Add terraform deployment example and documentation - (c7dff9f)
  • Filesystem store now delays before deleting temp file - (33d88c5)
  • Support deprecated symlink fields & fix bug for workers use CWD - (00431f9)
  • FastSlowStore now properly documented and used in LocalWorkerConfig - (728cb90)

🧪 Testing & CI

  • Fix ensure_full_copy_of_bytes_is_made_test flaky test (#528) - (14fdf4f)
  • Add small sleep in some tests to reduce flakes in CI (#526) - (fd4e6a3)
  • Mark nix-cargo and bazel tests as large ci instances (#524) - (a18d2d2)
  • Scale back a few CI runs (#516) - (245d9bb)
  • Add Kubernetes example (#479) - (e1c495f)
  • Avoid writer EOF until fast store complete (#480) - (2de8867)
  • Fix pre-commit hooks after 378b806 (#482) - (f2bd770)
  • Introduce Local Remote Execution (#471) - (449376b)
  • Separate CI runs by build system (#451) - (75a98f2)
  • Add remaining MacOS targets for further testing (#450) - (8f9da8f)
  • Fix MacOS tests (#449) - (befd1b6)
  • Give flaky memory store test more wiggle room (#448) - (ab0f1ac)
  • Add aarch64-apple-darwin to crates repository supported platforms (#440) - (ff6d5cf)
  • Fix pre-commit hooks after 3ec203b (#435) - (4aa2bc4)
  • Add pre-commit hooks for Starlark (#414) - (06654e6)
  • Add default pre-commit hooks (#405) - (228bdc4)
  • Add pre-commit infrastructure (#401) - (8de3014)
  • Added server readiness string listening to integration tests to reduce flakiness (#378) - (22abf90)
  • Refactor sanitizer CI (#344) - (ce64cc2)
  • Refactor Bazel unit test CI (#342) - (ef794c2)
  • Integration tests should work for Mac OS. (#334) - (1339e9d)
  • Make update_protos test compatible with Windows - (c2e2793)
  • Remove redundant formatting script - (93572d1)
  • Attempt to fix the flake Text file busy (os error 26) error in CI - (b730a90)
  • Attempt to fix flaky tests regarding Text file busy error - (637c8a9)
  • Fix flakey tests due to sync_all() not being called - (6c931fa)
  • Fix bug in BytestreamServer where it would ignore finish_write - (f645d69)
  • Files will now close if held open too long - (67b90e2)
  • Improve caching in CI and fix flakey prometheus test - (ea33b6c)
  • Fix incorrect type check. - (a22e437)
  • Add TSAN suppression and harness - (76326db)
  • Fix ASAN error and enable ASAN in CI - (e0cc0f9)
  • Add optional sanitizer build configurations - (a428e23)
  • Remove need for spawn in BytestreamServer - (44a4593)
  • Enable clippy by default for tests - (f211ef2)
  • Removes needless overoptimization of strings for DigestInfo - (4062d1d)
  • Move CI tests to run under docker - (5322c33)
  • Add convenience config to test clippy - (1185876)
  • Add a test for filestore loading from disk. - (5f3e9f5)
  • Remove the callbacks from the filesystem_store - (e2e62d2)
  • Adds GrpcStore and first integration tests - (117e173)
  • Fix bug in scheduler of not removing actions after execution - (f2b825b)
  • Fixes flakey filesystem_store_test - (717d87a)
  • First draft to get remote execution working - (f207dfa)
  • Restructure LocalWorker for easier testing - (d7d71a1)
  • Fix bug in memory store when receiving a zero byte object - (52445a1)
  • Fix github CI badge - (2758d22)
  • Adds automated CI tests on pull requests and master - (e647de0)
  • Add more basic scheduler support - (2edf514)
  • Dedup store will now bypass deduplication when size is small - (997be53)
  • Fix buf in bytestream_server when NotFound was returned - (a4634eb)
  • Upgrade rustc, use new nightly, rules_python, and rustfmt - (d0c31fb)
  • Fix format of util/tests/async_read_taker_test.rs - (cd12d1d)
  • dummy_test.sh will now print some equal signs when done - (1227d39)
  • Added single_item_wrong_digest_size test back to stable - (b517db1)
  • Add //:dummy_test that is useful for testing caching - (e5a1e9a)
  • Add dummy test that is used for easy caching - (efd449a)
  • Add test for bytestream::write() - (5dc8ac0)
  • Add bytestream server scaffolding - (7aff76f)
  • Add test for single item update action cache - (c3d89e1)
  • get_action_result done with tests - (fcc8a31)
  • Add first test for ac_server - (221ed5f)
  • Add test and fix bug when querying and using bad hash on .has() - (9adbe81)
  • Add test for batch_read_blobs - (4b1ae1a)
  • Add tests for invalid memory store requests - (4f8e5a7)
  • Add impl and tests for get store data - (7922f84)
  • Basic HashMap for memory store and enable store_one_item_existence test - (5206e74)
  • Add test for store_one_item_existence - (a6f1a70)
  • Add store and first test - (ed4bde4)
  • Add ability to resolve GetCapabilities and bazel connect testing - (1aba20c)

⚙️ Miscellaneous

  • Release nativelink 0.1.0 (#533) - (395674f)
  • MacOS use non darwin iconv (#534) - (2e4a131)
  • MacOS enable flake nix builds (#529) - (e1d35d6)
  • Mark GCP & AWS terraform experimental (#522) - (910ad03)
  • Decouple toolchain generation and execution hashes (#523) - (0be8a3a)
  • Support empty digests in stores (#507) - (41a85fb)
  • Increase the Max Blocking Threads (#510) - (6d07a86)
  • Remove beta toolchain from native Cargo builds (#514) - (334e738)
  • Migrate L2 announcements and LB-IPAM to cilium (#505) - (df6f5b9)
  • Move custom crates to nativelink dirs (#483) - (1aadd42)
  • Temporarily disable caching in image action (#466) - (a22150d)
  • Renames the project to nativelink (#463) - (2259251)
  • Remove caching from native cargo action (#458) - (4eab282)
  • Migrate logs to tracing (#387) - (c5c440a)
  • Implement API to drain a specific worker (#413) - (fbf1e44)
  • Use lowercase image names (#447) - (8b49bac)
  • Rename ExistanceStore to ExistanceCacheStore (#437) - (95f470c)
  • Remove the unnecessary scheduler.rs file (#431) - (dd17524)
  • Switch json5 to serde_json5 (#420) - (9279ba1)
  • Move action_messages & part of platform_properties to utils crate (#417) - (6bc991c)
  • Refactor ExistanceStore (#418) - (08b2954)
  • Remove OpenSSL from build requirements (#412) - (55a1292)
  • Remove openssl from flake (#411) - (05ec3f2)
  • Use Blake3 hasher for computing the action message (#395) - (9543964)
  • Rename Turbo Cache to Native Link (#402) - (7d9deaf)
  • Reduce allocations needed to hash uploads (#396) - (6e0d271)
  • Rename proto organization to trace_machina (#389) - (cf5f45b)
  • Migrate from lazy_static to once_cell (#377) - (c4b296c)
  • Move AWS terraform to it's own directory (#376) - (dd99eb0)
  • RBE consistency. (#364) - (7966d2d)
  • TLS Integration (#316) - (6c753d6)
  • Inline format args in cas_main (#337) - (62a2c1e)
  • Remove futures from scheduler_factory (#336) - (f15146d)
  • Globally enable windows symlinks (#327) - (9d08195)
  • propose a PR template to help newcomers get started more easily. - (b24b214)
  • Always make a copy in MemoryStore to reduce long-lasting allocs - (699dff9)
  • Refactor resource_info to support v2.3 resource names - (d3d0b64)
  • :get_part's writer is now a mutable reference. - (082a85c)
  • Future proof code from accidentially creating zombies - (4bd986c)
  • Remove spawn in dedup store - (59bbfd8)
  • Change Copyright to Trace Machina, Inc. - (d8685ed)
  • Support injecting property values into worker command. - (06c03de)
  • Populate the fast store on partial read - (e0e0a88)
  • Worker services now start after listeners to prevent error confiusion - (16f2ca5)
  • Output paths are relative to the work directory. - (f835e6d)
  • improve clarity for getting started. - (b677d78)
  • Create CODE_OF_CONDUCT.md - (a9709b4)
  • Implement fast slow metrics. - (10d2618)
  • Make stats in EvictingMap more useful - (03c1a07)
  • Removed properties should be known to the property manager. - (7d0999b)
  • Rename prometheus_utils to metrics_utils - (298bfb9)
  • Move Cargo.toml to Workspace and config to single library - (61d89cd)
  • Prometheus now publishes connected clients - (29fa44a)
  • Streams may now be resumed on disconnect - (b0b66a1)
  • Jobs now honor timeout in action - (fdc6d9b)
  • Replace Sha256 with Blake3 in DedupStore - (b30dfce)
  • Improve prometheus utils api - (87bd0e6)
  • Change StoreTrait::has to take &[DigestInfo]. - (732ae83)
  • Use Axum's router which supports multiple service types - (4856d35)
  • Create WaitExecution for ExecutionServer. - (857bf40)
  • Removes name from action state and make Operation::name schema - (0fac1ee)
  • Implementation of a cache lookup scheduler. - (fcdf192)
  • Move to UnorderedFutures in cas_store. - (5dc31e2)
  • Implement a GrpcScheduler to forward. - (bf0b933)
  • Promote chosen worker to distribute jobs. - (2fd1702)
  • Remove dependency on satackey/action-docker-layer-caching - (0801e26)
  • Implement Into traits indirectly via From - (86d8c60)
  • Scheduler is now an interface/trait - (13159ae)
  • Rename Scheduler to SimpleScheduler - (d9aeefc)
  • Goma places the platform properties in the command, so populate from there. - (69a9f5f)
  • Save action results in the action cache when they are completed on a worker. - (7bad256)
  • Move scheduler to use blocking mutexes - (74b2f04)
  • Change the version of C++ used for protoc to the earliest supported. - (b62338b)
  • Use recent Ubuntu with Clang as build environment - (5ac2c55)
  • Rename update_worker_with_internal_error - (58b5021)
  • Rename backends.rs to stores.rs - (451f68f)
  • Move trivial data access Mutex's to sync Mutex. - (812b94d)
  • Perform parallel reading of content directory to speed up startup. - (b354b25)
  • Remove obsolete raze configuration from Cargo.toml - (42b9ce0)
  • Kill currently running actions on scheduler disconnect. - (d5468dc)
  • Workers may be running multiple actions, so track them individually. - (99ee5b7)
  • Gzip compression is now disabled by default and configurable - (f4fcff2)
  • Scheduled jobs should be in a queue, not a stack. - (497446b)
  • Migrate from cargo_raze to crate_universe - (61309c1)
  • Simplify proto generation - (eebd6be)
  • Overhaul filesystem store to no longer use renameat2 - (a3cddf9)
  • Move from fast-async-mutex to async-lock crate as it's maintained. - (e172756)
  • Change license to Apache 2 license - (1147525)
  • Remove dependency on rust-nightly - (41028a9)
  • Enable Gzip compression support to GRPC - (438afbf)
  • Make JSON environmental variable lookup much better - (16a1a18)
  • Support more platforms for compilation - (7e09945)
  • Scheduler will retry on internal errors - (2be02e2)
  • LocalWorker will now purge the work directory on construction - (7325ffb)
  • Upgrade dependencies - (0765b63)
  • Upgrade LRU package and other deps - (1c02c09)
  • is_executable flag on files now overrides unix_mode exec flag - (420e7ee)
  • Remove the requirement of using reference for action_messages - (5daa4ff)
  • Services in main() now run in their own spawn - (09c52d9)
  • Implement execution_response() in WorkerApiServer - (e26549a)
  • Implement going_away() support to WorkerAPIServer - (0b68053)
  • Rename project turbo-cache - (76a7131)
  • Remove small item optimization in dedup_store and .has() calls - (f9e090d)
  • S3 now has a limit on number of active requests at a time - (025305c)
  • Cleanup un-needed lifetime annotations in store traits - (e341583)
  • Refactor code to use streams instead of AsyncRead/AsyncWrite - (697a11c)
  • Move evicting_map to use bytes instead of Vec for future efficencies - (9abc64c)
  • Minor change to make fastcdc return Bytes intead of BytesMut - (d5f1918)
  • EvictingMap is now a template - (791dc67)
  • .has() now returns size on success instead of boolean - (6a88752)
  • Slightly improve the performance of fastcdc by ~30% - (3063d2c)
  • Stores now support non-exact upload sizes & S3 store optimizations - (d093fdd)
  • Upgrade deps packages (tonic, tokio, tonic) - (692070a)
  • Various changes to improve debugging - (3a4d743)
  • Remove DigestInfo.trust_size & add expected_size to update() function - (e8a83eb)
  • Use fast-async-mutex for all mutex needs - (c5d450c)
  • Minor changes to ac_server to reduce chance of allocations - (41f989f)
  • Upgrade cargo packages and rust version - (7956a02)
  • Remove Sync requirement for ResultFuture - (59720c9)
  • Upgrade tonic, tokio and prost - (14eebc6)
  • MemoryStore now properly honors HASH and size constraints - (52b9ddb)
  • Move EvictingMapt to use DigestInfo - (1148952)
  • MemoryStore now can be configured to evict entries - (5830d0b)
  • ByteStream service now properly honors instance_name - (e9c8915)
  • Action Cache service now properly honors instance_name - (7af7cc6)
  • Remove need to pass CAS store to action cache service - (cc37b08)
  • CAS service now supports instance name partitioning - (faba6c9)
  • Moved hard coded settings to config - (0f7b2d0)
  • Configuration is now loaded from json - (99d170c)
  • Move store to use Pin and DigestInfo is no longer ref - (cf8cb79)
  • Now rust_fmt.sh will compile dependencies if needed - (c93b852)
  • Now use opt by default and add --config debug flag for dbg in bazel - (2abea44)
  • Better logging - (e61482a)
  • Improve logging - (7068b4b)
  • Store now has flag of when to verify the size of the digest - (6c70370)
  • Implement bufferstream's write() - (e09db45)
  • Move AC to inner functions - (5452f4b)
  • Upgrade tokio library - (3e8c210)
  • Implement get_action_result for action cache - (04d6a6d)
  • Remove need for exposing store in CasServer - (b4682cb)
  • Makes storage an Arc in prep for action cache good-ness - (365151a)
  • Cas batch_read_blobs now works - (42d4c8d)
  • Move some code to common util library - (2048d24)
  • Move macros to //util folder - (2fe5d30)
  • Cleanup capabilities a little - (4f5aad9)
  • CAS server moved to cas directory - (fcca6df)
  • Ignore target directory and ./rust_fmt.sh uses parallel now - (7a070b9)
  • Bazel remote execution server mock - (862a923)
  • Initial commit - (4bf7887)

⬆️ Bumps & Version Updates

  • Update dependency mintlify to v4.0.80 (#536) - (7564e5e)
  • Update Rust crate http to ^0.2.11 (#530) - (ca146ac)
  • Update native-cargo.yaml Runner Group (#511) - (e1843f1)
  • Update protobuf dependencies (#493) - (3dacdad)
  • Bump trivially bumpable deps (#488) - (96302cb)
  • Update protos after 1aadd42 (#489) - (9c6efe0)
  • Make max_bytes_per_stream optional in config (#474) - (a01a552)
  • Bump Rust version to 1.74 (#459) - (5412d7c)
  • Update nightly Rust toolchain for Bazel (#456) - (5acfa25)
  • Update Bazel to 6.4.0 (#381) - (2fb59b6)
  • Update Rust version to 1.73.0 (#371) - (56eda36)
  • Reduce flakiness of memory_store_test (#318) - (ee1f343)
  • Make memory_store_test compatible with Windows (#315) - (2c7e22b)
  • Update rules_rust to 0.29.0 - (d925e26)
  • Update Bazel to 6.3.2 - (c577db5)
  • Introduce get_part_ref() and migrate primary use to .get_part() - (fb6e1fd)
  • Update remote_execution.proto to v2.3 - (4c71336)
  • Update all dependencies to their latest versions - (6a72841)
  • Update Bazel to 6.2.1 - (d30571e)
  • Update dependencies. - (85bf34d)
  • Update rules_rust to 0.20.0 - (7a543c2)
  • Add minimum bazel version to .bazelversion - (a2be6f5)
  • Updates cargo packages - (a610e69)
  • Various minor changes - (2546a77)
  • Upgrade prost & tonic - (baad561)
  • Adds ability in proto for worker to return internal errors - (576aff4)
  • Restructure config format in prep for WorkerApiServer - (e8ce3a8)
  • Update Readme - (ecb4dc5)
  • Update third_party/Cargo in prep for s3_store - (f00b58f)
  • Update cargo packages - (786552f)
  • Move to 120 character line limit - (feb392d)
  • Update fully works for CAS w/ tests - (d2ed91e)