Releases: emmercm/igir
v3.0.1
Release Highlights
Just a quick release to update the chdman-js and maxcso-js npm packages in an effort to fix the Homebrew tap (emmercm/homebrew-igir#22). The two packages will now only download the binaries relevant to your OS & architecture, rather than downloading every prebuilt binary.
All changes
21 September 2024 - efaa97e - v3.0.1 (#1353)
21 September 2024 - b2fd2b7 - Chore: update chdman-js and maxcso-js (#1352)
4 September 2024 - 990fdcd - Docs: update (#1338)
2 September 2024 - aff53e4 - Docs: update (#1336)
31 August 2024 - 8a2e046 - CI: build GitHub pages on workflow dispatch (#1335)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
v3.0.0
See the project website for complete documentation, installation & usage instructions, and examples!
Release Highlights
This release has been a long time in the making, with more than 9,200 lines of code and documentation changed, and has taken significant testing—but it's finally ready!
Igir just passed the second birthday of its first release on August 24th. By the numbers, it has reached 7 community contributors, 344 stars on GitHub, and 18.2k downloads across all channels. I'm very proud of the project, and I'm humbled by the traction it's gotten with admittedly little advertisement. I will likely scale back the time I put into the project through the end of the year as this release has taken so much time & effort to get ready. Thank you all for your continued support!
The documentation website has had a significant refresh, but should still feel familiar. Thank you @netwaretcs for the new logo!
🚀 New major features
Support for reading & extracting CHD, CSO/ZSO/DAX, and NKit ISO files
Support for these common disk/disc compression formats is what prompted a major version change of Igir. And best yet—all of the required tools are bundled, so you don't have to source any of them! Thank you @maxexcloo, @unexpectedpanda, and @Djabal for all of your effort in development and general discussion!
Moved files are moved immediately
Previously, Igir would copy files from input to output during igir move, and then delete those "moved" files from the input directory as a cleanup action at the end. This was for the case when a single input file was used for different output files. Igir will now actually move files in real time, and it will then copy the first moved file in the output directory as needed for duplicates.
Copying zip files that don't need to be recreated
When not extracting (igir extract), Igir can now detect zip files that contain exactly the correct files for a game and copy that zip file as-is without recreating a new zip file. This helps preserve information such as zip comments and file timestamps.
🪛 New minor features
Breaking: --allow-excess-sets option
When not extracting or zipping, Igir would previously match archives that contained all of a game's files, even if the archive contained excess files. This is no longer the default behavior as it was likely unexpected, and instead, you must enable the behavior with the new --allow-excess-sets option. Thank you @PhasecoreX for the suggestion!
--input-checksum-quick option
The addition of new archive formats (above) that require bundled third-party tools to decompress may require a lot of temp file reading & writing. To bypass this, a new "quick" checksum option has been added that will only read checksums from archive headers and never extract contents to temp files. This comes with a lot of caveats, so please read the new documentation for more information.
--input-checksum-max <algorithm> option
The --input-checksum-min <algorithm> option which allows you to raise the minimum allowed checksum above CRC32 has existed since v2.6.0. It now has a sibling option --input-checksum-max <algorithm> which will let you constrain the maximum allowed checksum. This is useful because Igir will automatically enable the use of non-CRC32 algorithms in the case that DATs don't provide CRC32 checksums for all files.
🐛 Bug fixes, performance improvements
Support for DAT URLs to archives
@PhasecoreX found a bug/oversight where DAT URLs pointed to archive files wouldn't parse the archive's contents. This has been fixed!
More efficient rendering of progress bars
A change to render progress bars asynchronously should make all processing a little faster.
Cache file is tested after writing
When Igir writes the cache back to disk, it first writes it to a temp file and then overwrites the real location. That temp file will now be tested to make sure it was written correctly, which should reduce the chance of a corrupted file.
--dir-mirror with input subdirectories
The --dir-mirror option wasn't working as intended with input paths with path separators (i.e. / or \) in them. This should be fixed, and there is now an example highlighting the option's value.
Erroneous ROM header removal
No-Intro started including Clrmamepro-style header information in their headerless DATs, which was causing headers to be automatically removed when they shouldn't have been. This has been fixed.
⛓️💥 Breaking changes
Breaking: cache structure has significantly changed
Both the serialization format and the key structure of the cache file have been significantly changed, which will make your existing cache file invalid, so Igir will overwrite it with a fresh cache file. This is not the type of change that should need to happen often.
Breaking: all previously deprecated options and tokens have been removed
Several options that aren't visible in the help message have been deprecated for a long time, and all of them have been removed to keep the code tidy.
Breaking: --exclude-disks option
Now that Igir can parse CHDs, MAME games that specify "disks" will require their CHDs to be present to be considered a full match. This is because these games likely require their disks to function at all. To revert to the old behavior where disks aren't required, you can provide the --exclude-disks option.
Breaking: --prefer-ntsc and --prefer-pal have been removed
These options relied on naming schemes specific to certain DAT groups, and the tagging convention they relied on isn't present in the gross majority of games. These two options have been removed to avoid an expectation that they should work in most cases. You can use the --prefer-game-regex <pattern> option to recreate this behavior.
Breaking: --prefer-revision-newer and --prefer-revision-older combined into --prefer-revision <mode>
This brings the option in line with other options that have been added recently.
🫶 Sponsors
Thank you @dcramer who has been a long-time project sponsor, @lauripaulo for your recent sponsorship, and to everyone who has chosen to sponsor privately!
Every donation to the project is a blessing, and I'm thankful to everyone who spends their time making it a little better.
All changes
31 August 2024 - 2c53493 - v3.0.0 (#1334)
31 August 2024 - 21c7490 - Chore: update typescript-eslint monorepo to v8 (major) (#1323)
31 August 2024 - ea55b6a - CI: add .git-blame-ignore-revs (#1333)
31 August 2024 - 6746d8a - Refactor: sort modules into groups (#1332)
30 August 2024 - f7865f7 - Refactor: replace 'eslint-config-airbnb-typescript' with Prettier (#1331)
30 August 2024 - 79093af - Feature: v3.0.0 (#1233)
30 August 2024 - 9ea6039 - Fix: caching headers & signatures for archived files (#1330)
30 August 2024 - 3ed0202 - Docs: update (#1328)
30 August 2024 - cea8734 - Fix: dir mirror option when input path contains multiple path slugs (#1329)
30 August 2024 - 7b127e8 - Refactor: use the V8 serializer for the cache file (#1326)
29 August 2024 - b403669 - Refactor: rename progress bar icon names (#1327)
28 August 2024 - fcf0793 - Chore: update lockfile (#1325)
28 August 2024 - 27502fd - Docs: update asciinema (#1324)
28 August 2024 - 20dae09 - Fix: don't delay deleting input files during writing when moving (#1320)
28 Augus...
Contributors
Assets 8
v2.11.0
Release Highlights
Feature: cache ROM header & file signature detection
This will be an important performance boost ahead of various compressed disc support coming in v3. Thank you @dch82 for the suggestion!
Fixes: various performance bugs
- Checksums for files in the output directory are no longer calculated when not testing.
- Files being written to the same location (when the input and output directories are the same) are now skipped again, skipping expensive checksum calculations when testing.
Fixes: reducing concurrent file actions
- The concurrent filesize reading limit has been lowered to reduce disk thrash.
- Hashing of archive files when raw-copying now respects concurrent filesize read limits.
All changes
26 July 2024 - 3ec4a99 - v2.11.0 (#1240)
26 July 2024 - 54d4b73 - Feature: cache file signature detection (#1238)
26 July 2024 - 5d27f2b - Feature: cache ROM header detection (#1235)
25 July 2024 - e7c43e5 - Refactor: keep only a limited number of cache key mutexes (#1237)
25 July 2024 - c114188 - Fix: save the file cache on exit (#1236)
22 July 2024 - 29a3482 - Fix: File/ArchiveFile equality (#1232)
22 July 2024 - 216994d - Fix: don't throw if a single dat/input/patch directory is empty (#1231)
22 July 2024 - afb6449 - Fix: assume archives are full games when inferring DATs (#1230)
21 July 2024 - 1443fcd - Fix: don't CRC32 checksum files by default (#1229)
21 July 2024 - 9e40147 - Fix: reduce concurrent read & write file size limit (#1227)
21 July 2024 - 9d252bb - Fix: limit the concurrent file size being read when hashing raw archives (#1228)
20 July 2024 - 5d738ef - Refactor: consolidate file signature code (#1225)
20 July 2024 - 3e10abf - Fix: archive extension correction (#1224)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.10.1
Release Highlights
Feature: cleaning backup directory
A new --clean-backup <path> option has been added to move cleaned files to a backup directory rather than recycle or delete them. Thank you @maxexcloo for this suggestion!
All changes
19 July 2024 - d7d6222 - v2.10.1 (#1223)
19 July 2024 - f53ee63 - Docs: update (#1222)
19 July 2024 - fd0d355 - Feature: cleaning backup directory option (#1221)
17 July 2024 - 3c3f976 - Fix: ZJONSSON/node-unzipper#324 (#1220)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.10.0
Release Highlights
Feature: correction of ROM extensions
igir is most powerful when used in combination with DATs, but DATs aren't required to perform basic operations, and DATs don't always have complete information. For example, libretro's "metadats" don't include filename information for ROMs. Part of the filename can be assumed with the game name, but the extension can't be.
igir can now detect several common ROM formats by reading the file contents, and it will use the preferred ROM extension when information is missing (incomplete DATs as described above), or when the existing extension can't be trusted (when not using DATs). A new documentation page on ROM writing options has been added that details this behavior.
Fix: prevent output path collisions
@Djabal reported that some DATs include different ROMs with duplicate filenames. This would cause the same output file path to be written and rewritten with different ROMs, resulting in a non-deterministic outcome, and introducing the potential for crashes. igir will now prevent duplicate filenames from being written, stopping processing as soon as they're detected.
Options that sort games more granularly such as --dir-game-subdir always can be used to avoid filename collisions.
All changes
17 July 2024 - b4ead34 - v2.10.0 (#1219)
17 July 2024 - 5f30917 - Docs: update (#1218)
17 July 2024 - 351c91b - Feature: check for conflicting output file paths (#1208)
17 July 2024 - d02511c - Fix: enable CRC32 checksums when patch files are provided (#1217)
17 July 2024 - de38745 - Feature: correct bad ROM extensions (#1174)
15 July 2024 - 662ee57 - Chore: update Node.js to v20.15.1 (#1216)
14 July 2024 - 777b4d9 - Docs: update (#1214)
14 July 2024 - 8c7993d - Refactor: separate 'clean' help message options (#1215)
12 July 2024 - 0f0c9aa - CI: consistency with job permissions (#1212)
12 July 2024 - 865955a - Refactor: dir2dat use Game and ROM information from candidates (#1213)
12 July 2024 - 895ce43 - CI: update PR branch testing instructions on merge (#1211)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.9.2
Release Highlights
Fix: fixdats with output tokens
@Djabal found a bug where using fixdats and output tokens together may crash. This should be fixed.
Fix: BIOS DAT assumption
igir was assuming that a DAT with a single game but many ROMs should be treated as a BIOS DAT and should split all of those ROMs into individual games. This was to work around a poorly written libretro System.dat. @steveonjava found this to be a bad assumption, some consoles such as the NEC PC-88 have many ROMs per game, and may not have many games. This has been fixed.
All changes
11 July 2024 - 92d4f03 - v2.9.2 (#1210)
11 July 2024 - 0a212cf - Fix: remove duplicate games when combining DATs (#1209)
11 July 2024 - 7c300db - Fix: libretro System.dat BIOS assumption (#1207)
11 July 2024 - c555d9a - Fix: write fixdats to the output dir root (#1205)
10 July 2024 - 17b6860 - Feature: handle No-Intro PS3 BIOS parent/clone inference (#1206)
8 July 2024 - ec8d897 - Refactor: don't log the stack trace of expected errors (#1203)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.9.1
Release Highlights
Fix: downloading DAT URLs to a non-existent temp directory
@IDmedia found a fatal error introduced in v2.9.0 where igir would crash when downloading DAT URLs to a temporary directory that didn't already exist. This has been fixed.
All changes
4 July 2024 - 20a9ea7 - v2.9.1 (#1198)
4 July 2024 - b0e1011 - Fix: make multiple passes to delete empty directories (#1197)
3 July 2024 - 81c98e8 - Refactor: turn on the 'allowUnreachableCode' TypeScript option (#1194)
3 July 2024 - e50eee0 - Fix: mkdir before downloading DATs (#1196)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.9.0
Release Highlights
Feature/fix: allow exact-matching of archives
Some DATs such as the libretro BIOS System.dat catalog specific zip files, rather than whatever files are contained in that zip file. igir will now detect if any DAT files catalog specific archives and then calculate checksums as appropriate.
This additional checksum calculation can greatly increase the time to scan input ROM files, so an --input-checksum-archives 'never' option has been added to turn off this automatic behavior.
Thank you @IDmedia for the bug report!
Feature: temporary directory option
A new --temp-dir <path> option has been added to specify the directory used for temporary files. A new documentation page has been added to explain the option and the necessity of temporary files.
All changes
30 June 2024 - f6335a6 - v2.9.0 (#1193)
30 June 2024 - 2b3111f - Feature: allow exact-matching of archives in DATs (#1175)
30 June 2024 - 9245a31 - Feature: temp dir option (#1041)
28 June 2024 - eca2260 - Refactor: rename constants directory (#1192)
28 June 2024 - dda552b - Chore: update lockfile (#1191)
28 June 2024 - 860693d - Refactor: split constants into two classes (#1189)
27 June 2024 - f34f9d4 - Chore: update ESLint (#1190)
27 June 2024 - a608c77 - Chore: update dependency eslint-plugin-unicorn to v54 (#1186)
27 June 2024 - d58044f - Chore: update ESLint (#1183)
27 June 2024 - 715122c - Chore: update docs dependencies (#1182)
27 June 2024 - c218b5c - Refactor: fix issues with dependency updates (#1188)
27 June 2024 - c3f34d4 - Chore: update dev dependencies (#1185)
27 June 2024 - 9ec8b85 - Chore: update dependency typescript to v5.5.2 (#1187)
27 June 2024 - 16e260c - Chore: update dependency ts-jest to v29.1.5 (#1181)
27 June 2024 - 77d1d3b - Chore: update adRise/update-pr-branch action to v0.8.1 (#1184)
27 June 2024 - c6b238a - Chore: update Node.js to v20.15.0 (#1180)
23 June 2024 - 6ca0222 - Fix: logging supplied & parsed arguments (#1179)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.8.2
Release Highlights
Band-aid: don't crash with zip files >4.29GiB
An issue with the zip extraction library was causing fatal crashes with large zip files (e.g. Wii U games that are in the 8GiB+ range). The fatal crash is now prevented, but the library still isn't reading all compressed files correctly. An issue has been opened with the library and will continue to be monitored.
Thank you @tnvu and @keywal for helping diagnose this issue.
All changes
22 June 2024 - 7fdd2e7 - v2.8.2 (#1178)
22 June 2024 - 7c2c8cf - Fix: logging supplied & parsed arguments (#1177)
22 June 2024 - e0d3e21 - Fix: deferred ArchiveFile checksum waiting message (#1176)
22 June 2024 - 5ebee0c - Refactor: remove unused Options from CandidatePatchGenerator (#1173)
22 June 2024 - 12ce729 - Fix: ZJONSSON/node-unzipper#324 (#1171)
22 June 2024 - b28f248 - Fix: logging supplied & parsed arguments (#1172)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.8.1
Release Highlights
Feature: deferred checksum calculation
Checksums of archives that are being raw-copied (aren't being extracted, aren't being zipped) are now deferred until after 1G1R rules have been applied. This can save some significant processing time with large collections with large files.
Token updates
{mister}: added Acorn Archimedes, added TI-99/4A, fixed Atari 800, fixed Microsoft MSX1{pocket}: added GCE Vectrex{gameRegion}: renamed to{region}{gameLanguage}: renamed to{language}{genre}: newly added for DATs that provide this info (uncommon)- Updated console file extensions
- Reduced the log level on replacement errors
As always, see the output path tokens docs for complete information.
All changes
17 June 2024 - fc36582 - v2.8.1 (#1169)
16 June 2024 - 090069e - Feature: console token updates (#1168)
16 June 2024 - aa13e5e - Feature: add {genre} token (#1167)
16 June 2024 - f2b546c - Refactor: remove the "game" suffix from {gameRegion} and {gameLanguage} (#1166)
16 June 2024 - 3b715d6 - Fix: handle missing ROM filenames in clrmamepro DATs (#1165)
13 June 2024 - 2bdf92d - Refactor: remove "concurrently" language from module comments (#1164)
13 June 2024 - 81f0402 - Refactor: update gzip file signature (#1163)
13 June 2024 - e0c40bb - Refactor: defer ArchiveFile checksum calculation until after 1G1R applied (#1162)
13 June 2024 - 01d2138 - Refactor: increase file cache reaper frequency (#1161)
13 June 2024 - 1eae2b0 - Feature: console token updates (#1160)
13 June 2024 - d4db60a - Refactor: log token replacement errors at TRACE instead of WARN (#1159)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.