Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/v3.1.3 sprint 78 #473

Merged
merged 3 commits into from
Aug 22, 2023
Merged

Conversation

andrew-jameson
Copy link
Collaborator

Smithh-Co and others added 3 commits August 7, 2023 12:47
* added global temp directory to conftest

* Update conftest.py - linting
* - ADding t2 schema

* - small fixes to naming
- added t2.py
- Updating var names to match field names

* - Added new doc for T2
- Updated model fo rT2
- Added active parse function

* - Added T3 schema defs

* - Fixing most lint errors

* - Updated T3 to multirow record

* - Added unit test and data files
- updated field names

* - Updating var names to conform to DB max var name length

* - Updating based on unit test failure

* - adding datafile

* - Adding unit tests for t2 and t3

* - Fixed field types
- Updated test file

* - Removing many migrations to consolodate into one migration

* Revert "- Removing many migrations to consolodate into one migration"

This reverts commit 1455ae4.

* - Resolved test issues after merge

* - Resolve lint errors

* - Merged 1113.2

* - Removed unnecessary file

* - Rename model fields
- Combined migrations

* - fixed spelling in test

* - Fixed lint errors

* rm commented validators

* - Updated schema newlines to be consistent

* - Updated field names
- Updated tests
- Updated migration

* - consolodating migrations

* - Updated readme and backup script

* - Updated parse logic to batch create records

* - Fixed lint errors

* - Batching record serialization

* - Updated parse logic

* - Adding big s1 datafile

* - fixing lint errors

* - Removing test to see if its causing weird failure

* - Updating based on comments
- Removing big file since circi cant handle it

* - fixing error in update method

* - fixing error in update

* - Removed extraneous seek

* - updating ignore to ignore patch files

* - Made evaluate_trailer pure/immutable

* Revert "- Removing test to see if its causing weird failure"

This reverts commit 64b78b7.

* - Use custom transactions while parsing
- Handle transaction rollback on parse failure/error
- Update tests to use transaction logic
- Created new decorator to rollback database when assertion error is hit
- Making elastic search log level configurable
- Added test for inter parse rollback

* - updated store logic to one liner
- updated decorator to catch all exceptions
- updated tests

* - removed function
- renamed test

* - refactored multirecord schema to be a schema manager instead
- updated parse functions to use the new layout
- updated bulk create to manage batch size

* - Name update for clarity

* - Fix lint errors

* - Changing batch size to avoid crash in circi

* - Updated based on review
- Updated tests to reflect line number

* - Updated based on review/OH
- Added extra transaction safety

* - Updated view to show latest datafiles
- Added admin filter to show newest or all datafile records
- Updated indices to allow easier elastic queries

* - Updated based on review  comments

* - Updated based on feedback for preparse error handling
- updated tests

* - Updated search indices to have parent FK

* - Fix lint errors

* - Updated submission tests
- Moved create_datafile to util

* - fix lint errors

* - removing frontend filtering

* - addding datafile to admin model

* Revert "- addding datafile to admin model"

This reverts commit 35a6f24.

* - Fixed issue where datafile FK wasnt populating
- Regenerated migration

* - Readding datafile back to admin view now that the error is resolved

* - adding datafile back

* Revert "- Readding datafile back to admin view now that the error is resolved"

This reverts commit 2807425.

* - Removed unnecessary fields
- Updated dependencies
- Updated filter

* - Updated document to include required fields

* - Moved datafile reference to avoid confusion

* - Updating based on comments.
- Added base class to keep things a little more DRY

* - Refactored parsing to avoid custom transactions by leveraging the FK on the records. Rollbacks are a lot slower now, but hopefully that will happen much less versus successful parses.

* - small optimization for the rollback

* - Fix lint errors

* - Removing commented code

* - Installing build essentials

* Revert "- Installing build essentials"

This reverts commit 782a5f7.

* - adding setup tools and wheel

* Revert "- adding setup tools and wheel"

This reverts commit f529728.

* - Updating dependencies to get around pep issue

* - Pin factoryboy
- fix lint error

* - Updating manifest

* - Added EncryptedField class
- Updated schema's
- Updated datafiles since all are encrypted

* - Fix lint errors

* - Added decryption for ssp

* - Making the encrypted check stronger

* - Added section two models, schemas, and an initial test

* - Allowing fields to be nullable

* - add missing field

* - Consolodating migrations

* - Consolodate migration
- Add tests

* - Added encrypted field

* - Refactored fields and rowschema into seperate files
- Updated encrypted field to take decryption function

* - fix lint errors

* - Fixed file spacing

* - fix imports
- fix lint errors

* - Fix import error

* - Fix failing test

* - Revert erroneous change

* - Updating item numbers

* - Fixed item numbers

* - Updating to accomodate item number as string

* - Removed erroneous update that was causing error
- Fixed whitespace in datafiles

* - White space error

* - Resolved error

* - Fixing test

* - fix lint errors

* - updated fields to correct types

* - update race/ethnicity

* - Updating to the correct types

* - Resolve conflicts

* - Moving field updates to seperate migration to see if they take place

* Revert "- Moving field updates to seperate migration to see if they take place"

This reverts commit a3214f2.

* - Revert migration 15 to original state
- Generate new migration for field alterations

* - Fixed field names to correspond to model

---------

Co-authored-by: Jan Timpe <[email protected]>
Co-authored-by: Alex P <[email protected]>
Co-authored-by: Andrew <[email protected]>
Copy link
Collaborator

@ADPennington ADPennington left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

main update in Sprint 78 release is parsing of tanf section 2 data @lfrohlich @ttran-hub

@ADPennington ADPennington merged commit 2ef3699 into HHS:main Aug 22, 2023
2 checks passed
@ADPennington ADPennington deleted the release/v3.1.3-Sprint-78 branch August 22, 2023 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants