Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Initial implementation of HTTP connector #1649

Open
wants to merge 30 commits into
base: main
Choose a base branch
from

Conversation

@edgarrmondragon edgarrmondragon force-pushed the http-connector branch 2 times, most recently from 6ab2dc8 to 10ea58f Compare April 28, 2023 04:31
@edgarrmondragon edgarrmondragon changed the title Initial implementation of HTTP connector refactor: Initial implementation of HTTP connector Apr 28, 2023
@codecov
Copy link

codecov bot commented Apr 28, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.69%. Comparing base (39e1619) to head (911da30).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1649      +/-   ##
==========================================
+ Coverage   89.50%   89.69%   +0.18%     
==========================================
  Files          58       59       +1     
  Lines        4804     4872      +68     
  Branches      943      951       +8     
==========================================
+ Hits         4300     4370      +70     
+ Misses        351      350       -1     
+ Partials      153      152       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pnadolny13
Copy link
Contributor

pnadolny13 commented May 23, 2023

@edgarrmondragon how is this going? I'm just curious because once this merges I can update #1655 to use the same pattern you land on.

@edgarrmondragon edgarrmondragon marked this pull request as ready for review May 23, 2023 23:03
@edgarrmondragon edgarrmondragon requested review from a team and kgpayne as code owners May 23, 2023 23:03
@edgarrmondragon
Copy link
Collaborator Author

@edgarrmondragon how is this going? I'm just curious because once this merges I can update #1655 to use the same pattern you land on.

@pnadolny13 I think the basic approach is ready for review, but I'd like to read some thoughts from other engineers.

Copy link
Contributor

@kgpayne kgpayne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I really like this @edgarrmondragon 🙌 Added a few comments around the Connector API 🙂

self.connector = HTTPConnector(self.config)

# Override the connector's auth with the stream's auth
self.connector._auth = self.authenticator
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to make this auth override part of the connector constructor, rather than manipulating the private attr? Alternatively maybe a helper method to the connector class ?

Suggested change
self.connector._auth = self.authenticator
self.connector.override_auth(self.authenticator)

A public .authenticator attr could also work (wrapping the existing _auth private attr), but is maybe less self-documenting 🤔

singer_sdk/streams/rest.py Outdated Show resolved Hide resolved
singer_sdk/streams/rest.py Outdated Show resolved Hide resolved
singer_sdk/streams/rest.py Outdated Show resolved Hide resolved
Copy link

codspeed-hq bot commented Dec 7, 2023

CodSpeed Performance Report

Merging #1649 will not alter performance

Comparing http-connector (911da30) with main (39e1619)

Summary

✅ 6 untouched benchmarks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants