Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with AWS Permission on Header Object #52

Closed
mrufsvold opened this issue Jul 30, 2024 · 9 comments
Closed

Issue with AWS Permission on Header Object #52

mrufsvold opened this issue Jul 30, 2024 · 9 comments

Comments

@mrufsvold
Copy link

I have the following code:

using CloudStore: AWS
config = AWS.AWSCredentials()
AWS.get(
    "my valid s3 url copied directly from the S3 portal";
    service="S3",
    region="us-east-1",
    credentials = config
)

I've checked the credential object and everything looks good. Yet I am getting the following error:

ERROR: HTTP.Exceptions.StatusError(400, "GET", "/rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p%3D1.1.0/versioneddatasourceid_p%3D197/sourcefileid_p%3D1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet", HTTP.Messages.Response:
"""
HTTP/1.1 400 Bad Request
x-amz-request-id: 4RTEHTT2H8YZ1AY1
x-amz-id-2: EoMXqxx5Llr23ZFkiDE9CeCiDBaBH2sAsQiuItCEGZRh57gVrU+NFtracSos44nBZNFakfjVVerrShkdtcEHtavCB6dMafZMIECBFTkg43k=      
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Tue, 30 Jul 2024 21:08:05 GMT
Server: AmazonS3
Connection: close

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 's3' is wrong; expecting 'us-east-1'</Message><Region>us-east-1</Region><RequestId>4RTEHTT2H8YZ1AY1</RequestId><HostId>EoMXqxx5Llr23ZFkiDE9CeCiDBaBH2sAsQiuItCEGZRh57gVrU+NFtracSos44nBZNFakfjVVerrShkdtcEHtavCB6dMafZMIECBFTkg43k=</HostId></Error>""")
Stacktrace:
  [1] (::HTTP.ConnectionRequest.var"#connections#4"{…})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, socket_type_tls::Nothing, readtimeout::Int64, connect_timeout::Int64, 
logerrors::Bool, logtag::Nothing, closeimmediately::Bool, kw::@Kwargs{…})
    @ HTTP.ConnectionRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:144
  [2] connections
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:60 [inlined]
  [3] (::Base.var"#96#98"{…})(args::HTTP.Messages.Request; kwargs::@Kwargs{…})
    @ Base .\error.jl:296
  [4] (::HTTP.RetryRequest.var"#manageretries#3"{…})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_delays::ExponentialBackOff, retry_check::Function, retry_non_idempotent::Bool, kw::@Kwargs{…})
    @ HTTP.RetryRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:75
  [5] manageretries
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:30 [inlined]
  [6] (::HTTP.CookieRequest.var"#managecookies#4"{…})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::@Kwargs{…})
    @ HTTP.CookieRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:42
  [7] managecookies
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:19 [inlined]
  [8] (::HTTP.HeadersRequest.var"#defaultheaders#2"{…})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Nothing, 
basicauth::Bool, detect_content_type::Bool, canonicalize_headers::Bool, kw::@Kwargs{…})
    @ HTTP.HeadersRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:71
  [9] defaultheaders
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:14 [inlined]
 [10] (::HTTP.RedirectRequest.var"#redirects#3"{…})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::Nothing, kw::@Kwargs{…})
    @ HTTP.RedirectRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:25
 [11] redirects
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:14 [inlined]
 [12] (::CloudBase.var"#cloudmetrics#61"{…})(req::HTTP.Messages.Request; logexceptionalduration::Int64, kw::@Kwargs{…})       
    @ CloudBase C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:67
 [13] cloudmetrics
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:60 [inlined]
 [14] #1
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:122 [inlined]
 [15] (::HTTP.MessageRequest.var"#makerequest#3"{…})(method::String, url::URIs.URI, headers::Vector{…}, body::Vector{…}; copyheaders::Bool, response_stream::Nothing, http_version::HTTP.Strings.HTTPVersion, verbose::Int64, kw::@Kwargs{…})
    @ HTTP.MessageRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:35
 [16] makerequest
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:24 [inlined]
 [17] request(stack::HTTP.MessageRequest.var"#makerequest#3"{…}, method::String, url::String, h::Vector{…}, b::Vector{…}, q::Nothing; headers::Vector{…}, body::Vector{…}, query::Nothing, kw::@Kwargs{…})
    @ HTTP C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\HTTP.jl:457
 [18] request (repeats 2 times)
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
 [19] #get#5
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
 [20] top-level scope
    @ REPL[32]:1

caused by: HTTP.Exceptions.StatusError(400, "GET", "/rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p%3D1.1.0/versioneddatasourceid_p%3D197/sourcefileid_p%3D1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet", HTTP.Messages.Response:     
"""
HTTP/1.1 400 Bad Request
x-amz-request-id: 4RTEHTT2H8YZ1AY1
x-amz-id-2: EoMXqxx5Llr23ZFkiDE9CeCiDBaBH2sAsQiuItCEGZRh57gVrU+NFtracSos44nBZNFakfjVVerrShkdtcEHtavCB6dMafZMIECBFTkg43k=      
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Tue, 30 Jul 2024 21:08:05 GMT
Server: AmazonS3
Connection: close

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 's3' is wrong; expecting 'us-east-1'</Message><Region>us-east-1</Region><RequestId>4RTEHTT2H8YZ1AY1</RequestId><HostId>EoMXqxx5Llr23ZFkiDE9CeCiDBaBH2sAsQiuItCEGZRh57gVrU+NFtracSos44nBZNFakfjVVerrShkdtcEHtavCB6dMafZMIECBFTkg43k=</HostId></Error>""")
Stacktrace:
 [1] (::HTTP.ExceptionRequest.var"#exceptions#2"{…})(stream::HTTP.Streams.Stream{…}; status_exception::Bool, timedout::ConcurrentUtilities.TimedOut{…}, logerrors::Bool, logtag::Nothing, kw::@Kwargs{…})
   @ HTTP.ExceptionRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ExceptionRequest.jl:19
 [2] exceptions
   @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ExceptionRequest.jl:13 [inlined]
 [3] #2
   @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\TimeoutRequest.jl:22 [inlined]
 [4] macro expansion
   @ C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\try_with_timeout.jl:82 [inlined]
 [5] (::ConcurrentUtilities.var"#2#4"{…})()
   @ ConcurrentUtilities C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\ConcurrentUtilities.jl:9
Stacktrace:
  [1] try_yieldto(undo::typeof(Base.ensure_rescheduled))       
    @ Base .\task.jl:931
  [2] wait()
    @ Base .\task.jl:995
  [3] wait(c::Base.GenericCondition{ReentrantLock}; first::Bool)
    @ Base .\condition.jl:130
  [4] wait
    @ .\condition.jl:125 [inlined]
  [5] take_unbuffered(c::Channel{HTTP.Messages.Response})      
    @ Base .\channels.jl:494
  [6] take!
    @ .\channels.jl:471 [inlined]
  [7] try_with_timeout(f::Function, timeout::Int64, ::Type{HTTP.Messages.Response})
    @ ConcurrentUtilities C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\try_with_timeout.jl:89
  [8] (::HTTP.TimeoutRequest.var"#timeouts#3"{…})(stream::HTTP.Streams.Stream{…}; readtimeout::Int64, logerrors::Bool, logtag::Nothing, kw::@Kwargs{…})
    @ HTTP.TimeoutRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\TimeoutRequest.jl:21
  [9] (::HTTP.ConnectionRequest.var"#connections#4"{…})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, socket_type_tls::Nothing, readtimeout::Int64, connect_timeout::Int64, 
logerrors::Bool, logtag::Nothing, closeimmediately::Bool, kw::@Kwargs{…})
    @ HTTP.ConnectionRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:122
 [10] connections
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:60 [inlined]
 [11] (::Base.var"#96#98"{…})(args::HTTP.Messages.Request; kwargs::@Kwargs{…})
    @ Base .\error.jl:296
 [12] (::HTTP.RetryRequest.var"#manageretries#3"{…})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_delays::ExponentialBackOff, retry_check::Function, retry_non_idempotent::Bool, kw::@Kwargs{…})
    @ HTTP.RetryRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:75
 [13] manageretries
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:30 [inlined]
 [14] (::HTTP.CookieRequest.var"#managecookies#4"{…})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::@Kwargs{…})
    @ HTTP.CookieRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:42
 [15] managecookies
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:19 [inlined]
 [16] (::HTTP.HeadersRequest.var"#defaultheaders#2"{…})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Nothing, 
basicauth::Bool, detect_content_type::Bool, canonicalize_headers::Bool, kw::@Kwargs{…})
    @ HTTP.HeadersRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:71
 [17] defaultheaders
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:14 [inlined]
 [18] (::HTTP.RedirectRequest.var"#redirects#3"{…})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::Nothing, kw::@Kwargs{…})
    @ HTTP.RedirectRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:25
 [19] redirects
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:14 [inlined]
 [20] (::CloudBase.var"#cloudmetrics#61"{…})(req::HTTP.Messages.Request; logexceptionalduration::Int64, kw::@Kwargs{…})       
    @ CloudBase C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:67
 [21] cloudmetrics
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:60 [inlined]
 [22] #1
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:122 [inlined]
 [23] (::HTTP.MessageRequest.var"#makerequest#3"{…})(method::String, url::URIs.URI, headers::Vector{…}, body::Vector{…}; copyheaders::Bool, response_stream::Nothing, http_version::HTTP.Strings.HTTPVersion, verbose::Int64, kw::@Kwargs{…})
    @ HTTP.MessageRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:35
 [24] makerequest
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:24 [inlined]
 [25] request(stack::HTTP.MessageRequest.var"#makerequest#3"{…}, method::String, url::String, h::Vector{…}, b::Vector{…}, q::Nothing; headers::Vector{…}, body::Vector{…}, query::Nothing, kw::@Kwargs{…})
    @ HTTP C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\HTTP.jl:457
 [26] request (repeats 2 times)
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
 [27] #get#5
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
 [28] top-level scope
    @ REPL[32]:1
Some type information was truncated. Use `show(err)` to see complete types.

Here is my pkg status:

(TableConsolidator) pkg> status
Project TableConsolidator v0.1.0
Status `C:\Users\mrufsvold\Projects\DIL-price-transparency-psd\TableConsolidator.jl\Project.toml`
  [fbe9abb3] AWS v1.91.0
  [1c724243] AWSS3 v0.11.2
  [336ed68f] CSV v0.10.14
  [3365d9ee] CloudStore v1.6.1 `https://github.com/JuliaServices/CloudStore.jl.git#jq-http2`
  [b77c38ae] HTTP2 v1.0.0 `https://github.com/JuliaWeb/HTTP2.jl#main`
  [0f8b85d8] JSON3 v1.14.0
  [e6f89c97] LoggingExtras v1.0.3
  [98105f81] LoggingFormats v1.5.0
  [98572fba] Parquet2 v0.2.27
  [2dfb63ee] PooledArrays v1.4.3
  [20febd7b] Retry v0.4.1
  [bd369af6] Tables v1.12.0
  [28d57a85] Transducers v0.4.82
  [9d95f2ec] TypedTables v1.4.6
  [56ddb016] Logging
  [9a3f8284] Random

I'm running Julia 1.10.4 on windows 10

@mrufsvold
Copy link
Author

Trying to narrow in a bit:

config = CloudStore.AWS.AWSCredentials()
url ="https://ppad-processing-processed-datanode-output-prod.s3.amazonaws.com/rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p%3D1.1.0/versioneddatasourceid_p%3D197/sourcefileid_p%3D1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet"
region = "us-east-1"
nowarn = false

store, key = CloudStore.parseURLForDispatch(url, region, nowarn)
x = store
url = CloudStore.s3.makeURL(x, key)
headers=HTTP2.Headers()

CloudStore.AWS.head(url; headers, service="s3",
region=region,
credentials = config)

I wrote a script to isolate closer to the call that throws the error. The code above throws the same error. When I @edit the final call, it jumps me to CloudBase.AWS line 124:

HTTP.@client (first=(awslayer, cloudmetricslayer), last=()) (first=(), last=(cloudsignlayer,))

So I'm wondering if there is conflict between the HTTP2.Headers() and whatever HTTP.@client produces?

@mrufsvold
Copy link
Author

Sorry for spamming this, but I went to revisit this issue this morning, and I can no longer replicate the original error. Now, I'm getting a 403 forbidden error. I double checked the credential object, and it matches my .credential aws file.

When I dig a little deeper into the stack, I get a more informative error about the signature of the request not matching

julia> import CloudStore
julia> using HTTP2
julia> using AWS
julia> using AWSS3
julia> config = CloudStore.AWS.AWSCredentials()
julia> url ="https://ppad-processing-processed-datanode-output-prod.s3.amazonaws.com/rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p%3D1.1.0/versioneddatasourceid_p%3D197/sourcefileid_p%3D1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet"
julia> region = "us-east-1"
julia> nowarn = false
julia> store, key = CloudStore.parseURLForDispatch(url, region, nowarn)     
julia> x = store
julia> url = CloudStore.S3.API.makeURL(x, key)
julia> headers=HTTP2.Headers()
julia> CloudStore.API.getObject(x, url, headers; credentials=config)        

ERROR: HTTP.Exceptions.StatusError(403, "GET", "/rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p%3D1.1.0/versioneddatasourceid_p%3D197/sourcefileid_p%3D1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet", HTTP.Messages.Response:
"""
HTTP/1.1 403 Forbidden
x-amz-request-id: 7B92N9P287TVT1QR
x-amz-id-2: akdXcly64MzKGYIEc6ijFVubzYDDqQGXfnPsvYh0ZFxRUCK9fXO2D9pOpMM9CKmc9mv3n4y7lJM=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Wed, 31 Jul 2024 16:02:11 GMT
Server: AmazonS3

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>ASIA3WEQOA7MWBACO5UY</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
20240731T160212Z
20240731/us-east-1/s3/aws4_request
fb240e5adbed58f6b19b06e0c8362d18a2dacf9edb05fb5682e2552247365c5c</StringToSign><SignatureProvided>d643721ce38f2c98e424a805e99ba0ac824dc0633f14571a005cee0ebbe5bfdf</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 
2d 53 48 41 32 35 36 0a 32 30 32 34 30 37 33 31 54 31 36 30 32 31 32 5a 0a 32 30 32 34 30 37 33 31 2f 75 73 2d 65 61 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 66 62 32 34 30 65 35 61 64 62 65 64 35 38 66 36 
62 31 39 62 30 36 65 30 63 38 33 36 32 64 31 38 61 32 64 61 63 66 39 65 64 62 30 35 66 62 35 36 38 32 65 32 35 35 32 32 34 37 33 36 35 63 35 63</StringToSignBytes><CanonicalRequest>GET
/rollout202
⋮
7450-byte body
""")
Stacktrace:
  [1] (::HTTP.ConnectionRequest.var"#connections#4"{…})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, socket_type_tls::Nothing, readtimeout::Int64, connect_timeout::Int64, logerrors::Bool, logtag::Nothing, closeimmediately::Bool, kw::@Kwargs{…})
    @ HTTP.ConnectionRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:144
  [2] connections
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:60 [inlined]
  [3] (::Base.var"#96#98"{…})(args::HTTP.Messages.Request; kwargs::@Kwargs{…})
    @ Base .\error.jl:308
  [4] (::HTTP.RetryRequest.var"#manageretries#3"{…})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_delays::ExponentialBackOff, retry_check::Function, retry_non_idempotent::Bool, kw::@Kwargs{…})
    @ HTTP.RetryRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:75
  [5] manageretries
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:30 [inlined]
  [6] (::HTTP.CookieRequest.var"#managecookies#4"{…})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::@Kwargs{…})     
    @ HTTP.CookieRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:42
  [7] managecookies
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:19 [inlined]
  [8] (::HTTP.HeadersRequest.var"#defaultheaders#2"{…})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Nothing, basicauth::Bool, detect_content_type::Bool, canonicalize_headers::Bool, kw::@Kwargs{…})
    @ HTTP.HeadersRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:71
  [9] defaultheaders
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:14 [inlined]
 [10] (::HTTP.RedirectRequest.var"#redirects#3"{…})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::Nothing, kw::@Kwargs{…})
    @ HTTP.RedirectRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:25
 [11] redirects
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:14 [inlined]
 [12] (::CloudBase.var"#cloudmetrics#61"{…})(req::HTTP.Messages.Request; logexceptionalduration::Int64, kw::@Kwargs{…})
    @ CloudBase C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:67
 [13] cloudmetrics
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:60 [inlined]
 [14] #1
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:122 [inlined]
 [15] (::HTTP.MessageRequest.var"#makerequest#3"{…})(method::String, url::URIs.URI, headers::Vector{…}, body::Vector{…}; copyheaders::Bool, response_stream::Nothing, http_version::HTTP.Strings.HTTPVersion, verbose::Int64, kw::@Kwargs{…})
    @ HTTP.MessageRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:35
 [16] makerequest
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:24 [inlined]
 [17] request(stack::HTTP.MessageRequest.var"#makerequest#3"{…}, method::String, url::String, h::Vector{…}, b::Vector{…}, q::Nothing; headers::Vector{…}, body::Vector{…}, query::Nothing, kw::@Kwargs{…})
    @ HTTP C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\HTTP.jl:457    
 [18] request (repeats 2 times)
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
 [19] get
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
 [20] #getObject#5
    @ C:\Users\mrufsvold\.julia\packages\CloudStore\qT25C\src\s3.jl:30 [inlined]
 [21] top-level scope
    @ REPL[45]:1

caused by: HTTP.Exceptions.StatusError(403, "GET", "/rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p%3D1.1.0/versioneddatasourceid_p%3D197/sourcefileid_p%3D1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet", HTTP.Messages.Response:
"""
HTTP/1.1 403 Forbidden
x-amz-request-id: 7B92N9P287TVT1QR
x-amz-id-2: akdXcly64MzKGYIEc6ijFVubzYDDqQGXfnPsvYh0ZFxRUCK9fXO2D9pOpMM9CKmc9mv3n4y7lJM=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Wed, 31 Jul 2024 16:02:11 GMT
Server: AmazonS3

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>ASIA3WEQOA7MWBACO5UY</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
20240731T160212Z
20240731/us-east-1/s3/aws4_request
fb240e5adbed58f6b19b06e0c8362d18a2dacf9edb05fb5682e2552247365c5c</StringToSign><SignatureProvided>d643721ce38f2c98e424a805e99ba0ac824dc0633f14571a005cee0ebbe5bfdf</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 
2d 53 48 41 32 35 36 0a 32 30 32 34 30 37 33 31 54 31 36 30 32 31 32 5a 0a 32 30 32 34 30 37 33 31 2f 75 73 2d 65 61 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 66 62 32 34 30 65 35 61 64 62 65 64 35 38 66 36 
62 31 39 62 30 36 65 30 63 38 33 36 32 64 31 38 61 32 64 61 63 66 39 65 64 62 30 35 66 62 35 36 38 32 65 32 35 35 32 32 34 37 33 36 35 63 35 63</StringToSignBytes><CanonicalRequest>GET
/rollout202
⋮
7450-byte body
""")
Stacktrace:
 [1] (::HTTP.ExceptionRequest.var"#exceptions#2"{…})(stream::HTTP.Streams.Stream{…}; status_exception::Bool, timedout::ConcurrentUtilities.TimedOut{…}, 
logerrors::Bool, logtag::Nothing, kw::@Kwargs{…})
   @ HTTP.ExceptionRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ExceptionRequest.jl:19
 [2] exceptions
   @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ExceptionRequest.jl:13 [inlined]
 [3] #2
   @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\TimeoutRequest.jl:22 [inlined]
 [4] macro expansion
   @ C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\try_with_timeout.jl:82 [inlined]
 [5] (::ConcurrentUtilities.var"#2#4"{…})()
   @ ConcurrentUtilities C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\ConcurrentUtilities.jl:9
Stacktrace:
  [1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
    @ Base .\task.jl:931
  [2] wait()
    @ Base .\task.jl:995
  [3] wait(c::Base.GenericCondition{ReentrantLock}; first::Bool)
    @ Base .\condition.jl:130
  [4] wait
    @ .\condition.jl:125 [inlined]
  [5] take_unbuffered(c::Channel{HTTP.Messages.Response})
    @ Base .\channels.jl:494
  [6] take!
    @ .\channels.jl:471 [inlined]
  [7] try_with_timeout(f::Function, timeout::Int64, ::Type{HTTP.Messages.Response})
    @ ConcurrentUtilities C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\try_with_timeout.jl:89
  [8] (::HTTP.TimeoutRequest.var"#timeouts#3"{…})(stream::HTTP.Streams.Stream{…}; readtimeout::Int64, logerrors::Bool, logtag::Nothing, kw::@Kwargs{…}) 
    @ HTTP.TimeoutRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\TimeoutRequest.jl:21
  [9] (::HTTP.ConnectionRequest.var"#connections#4"{…})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, socket_type_tls::Nothing, readtimeout::Int64, connect_timeout::Int64, logerrors::Bool, logtag::Nothing, closeimmediately::Bool, kw::@Kwargs{…})
    @ HTTP.ConnectionRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:122
 [10] connections
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:60 [inlined]
 [11] (::Base.var"#96#98"{…})(args::HTTP.Messages.Request; kwargs::@Kwargs{…})
    @ Base .\error.jl:308
 [12] (::HTTP.RetryRequest.var"#manageretries#3"{…})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_delays::ExponentialBackOff, retry_check::Function, retry_non_idempotent::Bool, kw::@Kwargs{…})
    @ HTTP.RetryRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:75
 [13] manageretries
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:30 [inlined]
 [14] (::HTTP.CookieRequest.var"#managecookies#4"{…})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::@Kwargs{…})     
    @ HTTP.CookieRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:42
 [15] managecookies
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:19 [inlined]
 [16] (::HTTP.HeadersRequest.var"#defaultheaders#2"{…})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Nothing, basicauth::Bool, detect_content_type::Bool, canonicalize_headers::Bool, kw::@Kwargs{…})
    @ HTTP.HeadersRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:71
 [17] defaultheaders
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:14 [inlined]
 [18] (::HTTP.RedirectRequest.var"#redirects#3"{…})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::Nothing, kw::@Kwargs{…})
    @ HTTP.RedirectRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:25
 [19] redirects
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:14 [inlined]
 [20] (::CloudBase.var"#cloudmetrics#61"{…})(req::HTTP.Messages.Request; logexceptionalduration::Int64, kw::@Kwargs{…})
    @ CloudBase C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:67
 [21] cloudmetrics
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:60 [inlined]
 [22] #1
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:122 [inlined]
 [23] (::HTTP.MessageRequest.var"#makerequest#3"{…})(method::String, url::URIs.URI, headers::Vector{…}, body::Vector{…}; copyheaders::Bool, response_stream::Nothing, http_version::HTTP.Strings.HTTPVersion, verbose::Int64, kw::@Kwargs{…})
    @ HTTP.MessageRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:35
 [24] makerequest
    @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:24 [inlined]
 [25] request(stack::HTTP.MessageRequest.var"#makerequest#3"{…}, method::String, url::String, h::Vector{…}, b::Vector{…}, q::Nothing; headers::Vector{…}, body::Vector{…}, query::Nothing, kw::@Kwargs{…})
    @ HTTP C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\HTTP.jl:457    
 [26] request (repeats 2 times)
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
 [27] get
    @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
 [28] #getObject#5
    @ C:\Users\mrufsvold\.julia\packages\CloudStore\qT25C\src\s3.jl:30 [inlined]
 [29] top-level scope
    @ REPL[45]:1
Some type information was truncated. Use `show(err)` to see complete types. 

@mrufsvold mrufsvold changed the title Malformed Auth Header Error Issue with AWS Permission on Header Object Aug 1, 2024
@mrufsvold
Copy link
Author

mrufsvold commented Aug 1, 2024

Renaming because I was going down the wrong path.

I switched my CloudStore.jl version to the current release (to rule out changes made by HTTP2.jl). And I get the same error even after the change. However, when I use AWSS3.jl, I can access the file just fine.

import CloudStore
using AWS
using AWSS3

config = CloudStore.AWS.AWSCredentials()
url ="https://ppad-processing-processed-datanode-output-prod.s3.amazonaws.com/rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p%3D1.1.0/versioneddatasourceid_p%3D197/sourcefileid_p%3D1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet"
region = "us-east-1"
CloudStore.get(url;
    service="S3",
    region=region,
    credentials = config
) 
# Throws a Status error with 403 forbidden
# I get the same behavior if I don't pass any kwargs


p = S3Path("s3://ppad-processing-processed-datanode-output-prod/rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p=1.1.0/versioneddatasourceid_p=197/sourcefileid_p=1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet")
open(p) 
# Returns a FileBuffer 

At this point, I'm reasonably confident that this is user error, but I'm not sure how to go about fixing it from the docs.

EDIT: Looking at this StackOverflow question, maybe it's the leading / in the key of the GET request? But if that's the case, then I would have expected this to affect other users before me!

@quinnj
Copy link
Member

quinnj commented Aug 2, 2024

It seems to be the encoding of the = in the object name:

have: GET\n/rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p%253D1.1.0/versioneddatasourceid_p%253D197/sourcefileid_p%253D1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet\n\naccept:*/*\naccept-encoding:gzip\nhost:ppad-processing-processed-datanode-output-prod-jq.s3.us-east-2.amazonaws.com\nuser-agent:HTTP2.jl/1.10.4\nx-amz-date:20240802T193952Z\n\naccept;accept-encoding;host;user-agent;x-amz-date\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
want: GET\n/rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p%3D1.1.0/versioneddatasourceid_p%3D197/sourcefileid_p%3D1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet\n\naccept:*/*\naccept-encoding:gzip\nhost:ppad-processing-processed-datanode-output-prod-jq.s3.us-east-2.amazonaws.com\nuser-agent:HTTP2.jl/1.10.4\nx-amz-date:20240802T193952Z\n\naccept;accept-encoding;host;user-agent;x-amz-date\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

let me double check how = is getting handled and why we're encoding it differently than aws.

@quinnj
Copy link
Member

quinnj commented Aug 2, 2024

Oh wait, so the url you're passing to CloudStore.get already has the = encoded as %3D, so it's getting double-encoded when you make the get request. You should just use the plain url name w/o encoding and CloudStore will do the encoding for you. So this should work:

using CloudBase: AWS
config = CloudStore.AWS.AWSCredentials()
bucket = AWS.Bucket("ppad-processing-processed-datanode-output-prod", "us-east-1")
file = CloudStore.get(
    bucket,
    "rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p=1.1.0/versioneddatasourceid_p=197/sourcefileid_p=1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet"
    ; credentials=config
)

@mrufsvold
Copy link
Author

mrufsvold commented Aug 5, 2024

I am no longer getting a 403! Thanks for the catch, @quinnj. Feels a little silly that the URL listed on the S3 page is pre-encoded.

However, I'm now getting a MethodError pretty far down the call stack about the IOBuffer allocated for the request.

using CloudBase: AWS
config = CloudStore.AWS.AWSCredentials()
bucket = AWS.Bucket("ppad-processing-processed-datanode-output-prod", "us-east-1")
file = CloudStore.get(
    bucket,
    "rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p=1.1.0/versioneddatasourceid_p=197/sourcefileid_p=1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet"
    ; credentials=config
)

## Throws
ERROR: TaskFailedException

    nested task error: HTTP.RequestError:
    HTTP.Request:
    HTTP.Messages.Request:
    """
    GET /rollout2024q3-197/rollout2024q3-197_innetwork/data_version_p=1.1.0/versioneddatasourceid_p=197/sourcefileid_p=1605809/06c4b233-fa63-4bd4-b053-37b7a80aae35.parquet HTTP/1.1
    Range: bytes=0-14936
    Host: ppad-processing-processed-datanode-output-prod.s3.us-east-1.amazonaws.com
    Accept: */*
    User-Agent: HTTP.jl/1.10.4
    Content-Length: 0
    Accept-Encoding: gzip
    x-amz-date: 20240805T141450Z
    x-amz-security-token: IQoJb3JpZ2luX2VjEO///////////wEaCXVzLWVhc3QtMSJIMEYCIQDMLAFth54RBcyj6/z9uWscL8Um0qhwfCSmZoSoQzpNUQIhAKmeT1ChJtDnsTYUCFc3pGfXZiE56+oRQYdgVrgRW21+Kq0DCNf//////////wEQAxoMODAzNDYxNzkzNzUzIgz0HSfLsFKooXFu6oUqgQNoVFF8ggXbMi68/KUWiDQbQ2bs3PBbiUyRJQECEIBVnxviB3JiIEn0uNF5go3fA0/Ji+FW2FGhHJfGYdnMYO3EORmfo76IodYZTUOAaIhaAz1DK0M3A2Xr8H0GZ2RNGZzTXzWU5W0uwab82sIiBkJpQF7fwa4TDRIkVQcAp4rhW4G4cnmWBUS9VJq4wF8HPWejJDJAyNOL9NtW7EblBWkrKBGeXkuOPyouv60Xc1z21zEI4A+BK8MI4UgEnXZEmdfUk1MjQ4oL3LAaPCXmehYgo6p/TZNIVHuIIQbAG5vw5Q7PBvAOQh5WUI7//i/IbjeS348rVsPoyJ+gISX6K8rSl4jStcngBmlv+qTDZcUS8uN7hblUUZLdFSAJQkkRWLiynTYYbFsFdht1J4BvPNUERw2QCgPJZN65iVStvGI+2a72Toiu78jW4xWhgXmTManV45y/5cImU44cRlfP2ASjvwLpUwtWZzBDqXsakmPLbTlY2UT6wbRukoMmg3I0Dc2zMLy5w7UGOqUBqCwetz+SdiQXlyrk/qZEb+6q8ZvJrCbiMIvKH4iE4dCuanKxr5q9xu8ewCbsXxW95p1N+Em0lAL+mzoP9drE/t6bKgw/rruN9PY2E7KEtN1ATjC9rPAL3HqFSgeudfJ7UvN220imGIr94KUcJdCqR7qNK2wf87q7mh4khxZXb9qCGJB58qBMr6kLmceGGRH8Nzqfh9bbY/51dhytmAZgSMoZx8/I
    x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
    Authorization: ******

    """Underlying error:
    TaskFailedException

        nested task error: MethodError: no method matching alloc_request(::Base.GenericIOBuffer{SubArray{UInt8, 1, Vector{UInt8}, Tuple{UnitRange{Int64}}, true}}, ::UInt64)

        Closest candidates are:
          alloc_request(::IOBuffer, ::UInt64)
           @ Base stream.jl:605

        Stacktrace:
         [1] readbytes!(http::HTTP.Streams.Stream{…}, buf::Base.GenericIOBuffer{…}, n::Int64)
           @ HTTP.Streams C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\Streams.jl:283
         [2] readall!(http::HTTP.Streams.Stream{…}, buf::Base.GenericIOBuffer{…})
           @ HTTP.Streams C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\Streams.jl:311
         [3] readbody!(stream::HTTP.Streams.Stream{…}, res::HTTP.Messages.Response, buf_or_stream::HTTP.Streams.Stream{…}, lock::ReentrantLock)
           @ HTTP.StreamRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\StreamRequest.jl:164
         [4] readbody(stream::HTTP.Streams.Stream{…}, res::HTTP.Messages.Response, decompress::Nothing, lock::ReentrantLock)
           @ HTTP.StreamRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\StreamRequest.jl:142
         [5] macro expansion
           @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\StreamRequest.jl:54 [inlined]
         [6] (::HTTP.StreamRequest.var"#3#5"{})()
           @ HTTP.StreamRequest C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\ConcurrentUtilities.jl:9
    Stacktrace:
      [1] (::HTTP.ConnectionRequest.var"#connections#4"{})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, socket_type_tls::Nothing, readtimeout::Int64, connect_timeout::Int64, logerrors::Bool, logtag::Nothing, closeimmediately::Bool, kw::@Kwargs{})
        @ HTTP.ConnectionRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:143
      [2] connections
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:60 [inlined]
      [3] (::Base.var"#96#98"{})(args::HTTP.Messages.Request; kwargs::@Kwargs{})
        @ Base .\error.jl:296
      [4] (::HTTP.RetryRequest.var"#manageretries#3"{})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_delays::ExponentialBackOff, retry_check::Function, retry_non_idempotent::Bool, kw::@Kwargs{})
        @ HTTP.RetryRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:75
      [5] manageretries
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:30 [inlined]
      [6] (::HTTP.CookieRequest.var"#managecookies#4"{})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::@Kwargs{}) 
        @ HTTP.CookieRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:42
      [7] managecookies
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:19 [inlined]
      [8] (::HTTP.HeadersRequest.var"#defaultheaders#2"{})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Nothing, basicauth::Bool, detect_content_type::Bool, canonicalize_headers::Bool, kw::@Kwargs{})
        @ HTTP.HeadersRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:71
      [9] defaultheaders
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:14 [inlined]
     [10] (::HTTP.RedirectRequest.var"#redirects#3"{})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::SubArray{…}, kw::@Kwargs{})
        @ HTTP.RedirectRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:25
     [11] redirects
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:14 [inlined]
     [12] (::CloudBase.var"#cloudmetrics#61"{})(req::HTTP.Messages.Request; logexceptionalduration::Int64, kw::@Kwargs{})
        @ CloudBase C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:67
     [13] cloudmetrics
        @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:60 [inlined]
     [14] #1
        @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:122 [inlined]
     [15] (::HTTP.MessageRequest.var"#makerequest#3"{})(method::String, url::URIs.URI, headers::Vector{…}, body::Vector{…}; copyheaders::Bool, response_stream::SubArray{…}, http_version::HTTP.Strings.HTTPVersion, verbose::Int64, kw::@Kwargs{})
        @ HTTP.MessageRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:35
     [16] makerequest
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:24 [inlined]
     [17] request(stack::HTTP.MessageRequest.var"#makerequest#3"{}, method::String, url::String, h::Vector{…}, b::Vector{…}, q::Nothing; headers::Vector{…}, body::Vector{…}, query::Nothing, kw::@Kwargs{})
        @ HTTP C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\HTTP.jl:457     [18] request (repeats 2 times)
        @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
     [19] get
        @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
     [20] #getObject#5
        @ C:\Users\mrufsvold\.julia\packages\CloudStore\6XBiA\src\s3.jl:30 [inlined]
     [21] (::CloudStore.API.var"#26#28"{})()
        @ CloudStore.API C:\Users\mrufsvold\.julia\packages\CloudStore\6XBiA\src\get.jl:228

    caused by: TaskFailedException

        nested task error: MethodError: no method matching alloc_request(::Base.GenericIOBuffer{SubArray{UInt8, 1, Vector{UInt8}, Tuple{UnitRange{Int64}}, true}}, ::UInt64)

        Closest candidates are:
          alloc_request(::IOBuffer, ::UInt64)
           @ Base stream.jl:605

        Stacktrace:
         [1] readbytes!(http::HTTP.Streams.Stream{…}, buf::Base.GenericIOBuffer{…}, n::Int64)
           @ HTTP.Streams C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\Streams.jl:283
         [2] readall!(http::HTTP.Streams.Stream{…}, buf::Base.GenericIOBuffer{…})
           @ HTTP.Streams C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\Streams.jl:311
         [3] readbody!(stream::HTTP.Streams.Stream{…}, res::HTTP.Messages.Response, buf_or_stream::HTTP.Streams.Stream{…}, lock::ReentrantLock)
           @ HTTP.StreamRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\StreamRequest.jl:164
         [4] readbody(stream::HTTP.Streams.Stream{…}, res::HTTP.Messages.Response, decompress::Nothing, lock::ReentrantLock)
           @ HTTP.StreamRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\StreamRequest.jl:142
         [5] macro expansion
           @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\StreamRequest.jl:54 [inlined]
         [6] (::HTTP.StreamRequest.var"#3#5"{})()
           @ HTTP.StreamRequest C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\ConcurrentUtilities.jl:9
    Stacktrace:
      [1] sync_end(c::Channel{Any})
        @ Base .\task.jl:448
      [2] macro expansion
        @ .\task.jl:480 [inlined]
      [3] streamlayer(stream::HTTP.Streams.Stream{…}; iofunction::Nothing, decompress::Nothing, logerrors::Bool, logtag::Nothing, timedout::ConcurrentUtilities.TimedOut{…}, kw::@Kwargs{})
        @ HTTP.StreamRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\StreamRequest.jl:35
      [4] streamlayer
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\StreamRequest.jl:21 [inlined]
      [5] (::CloudBase.var"#cloudsign#64"{})(stream::HTTP.Streams.Stream{…}; aws::Bool, awsv2::Bool, azure::Bool, kw::@Kwargs{})
        @ CloudBase C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:105
      [6] cloudsign
        @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:97 [inlined]
      [7] (::HTTP.ExceptionRequest.var"#exceptions#2"{})(stream::HTTP.Streams.Stream{…}; status_exception::Bool, timedout::ConcurrentUtilities.TimedOut{…}, logerrors::Bool, logtag::Nothing, kw::@Kwargs{})
        @ HTTP.ExceptionRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ExceptionRequest.jl:14
      [8] exceptions
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ExceptionRequest.jl:13 [inlined]
      [9] #2
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\TimeoutRequest.jl:22 [inlined]
     [10] macro expansion
        @ C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\try_with_timeout.jl:82 [inlined]
     [11] (::ConcurrentUtilities.var"#2#4"{})()
        @ ConcurrentUtilities C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\ConcurrentUtilities.jl:9
    Stacktrace:
      [1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
        @ Base .\task.jl:931
      [2] wait()
        @ Base .\task.jl:995
      [3] wait(c::Base.GenericCondition{ReentrantLock}; first::Bool)        
        @ Base .\condition.jl:130
      [4] wait
        @ .\condition.jl:125 [inlined]
      [5] take_unbuffered(c::Channel{HTTP.Messages.Response})
        @ Base .\channels.jl:494
      [6] take!
        @ .\channels.jl:471 [inlined]
      [7] try_with_timeout(f::Function, timeout::Int64, ::Type{HTTP.Messages.Response})
        @ ConcurrentUtilities C:\Users\mrufsvold\.julia\packages\ConcurrentUtilities\QOkoO\src\try_with_timeout.jl:89
      [8] (::HTTP.TimeoutRequest.var"#timeouts#3"{})(stream::HTTP.Streams.Stream{…}; readtimeout::Int64, logerrors::Bool, logtag::Nothing, kw::@Kwargs{})
        @ HTTP.TimeoutRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\TimeoutRequest.jl:21
      [9] (::HTTP.ConnectionRequest.var"#connections#4"{})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, socket_type_tls::Nothing, readtimeout::Int64, connect_timeout::Int64, logerrors::Bool, logtag::Nothing, closeimmediately::Bool, kw::@Kwargs{})
        @ HTTP.ConnectionRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:122
     [10] connections
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\ConnectionRequest.jl:60 [inlined]
     [11] (::Base.var"#96#98"{})(args::HTTP.Messages.Request; kwargs::@Kwargs{})
        @ Base .\error.jl:296
     [12] (::HTTP.RetryRequest.var"#manageretries#3"{})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_delays::ExponentialBackOff, retry_check::Function, retry_non_idempotent::Bool, kw::@Kwargs{})
        @ HTTP.RetryRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:75
     [13] manageretries
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RetryRequest.jl:30 [inlined]
     [14] (::HTTP.CookieRequest.var"#managecookies#4"{})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::@Kwargs{}) 
        @ HTTP.CookieRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:42
     [15] managecookies
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\CookieRequest.jl:19 [inlined]
     [16] (::HTTP.HeadersRequest.var"#defaultheaders#2"{})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Nothing, basicauth::Bool, detect_content_type::Bool, canonicalize_headers::Bool, kw::@Kwargs{})
        @ HTTP.HeadersRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:71
     [17] defaultheaders
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\HeadersRequest.jl:14 [inlined]
     [18] (::HTTP.RedirectRequest.var"#redirects#3"{})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::SubArray{…}, kw::@Kwargs{})
        @ HTTP.RedirectRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:25
     [19] redirects
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\RedirectRequest.jl:14 [inlined]
     [20] (::CloudBase.var"#cloudmetrics#61"{})(req::HTTP.Messages.Request; logexceptionalduration::Int64, kw::@Kwargs{})
        @ CloudBase C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:67
     [21] cloudmetrics
        @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:60 [inlined]
     [22] #1
        @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:122 [inlined]
     [23] (::HTTP.MessageRequest.var"#makerequest#3"{})(method::String, url::URIs.URI, headers::Vector{…}, body::Vector{…}; copyheaders::Bool, response_stream::SubArray{…}, http_version::HTTP.Strings.HTTPVersion, verbose::Int64, kw::@Kwargs{})
        @ HTTP.MessageRequest C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:35
     [24] makerequest
        @ C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\clientlayers\MessageRequest.jl:24 [inlined]
     [25] request(stack::HTTP.MessageRequest.var"#makerequest#3"{}, method::String, url::String, h::Vector{…}, b::Vector{…}, q::Nothing; headers::Vector{…}, body::Vector{…}, query::Nothing, kw::@Kwargs{})
        @ HTTP C:\Users\mrufsvold\.julia\packages\HTTP\sJD5V\src\HTTP.jl:457     [26] request (repeats 2 times)
        @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
     [27] get
        @ C:\Users\mrufsvold\.julia\packages\CloudBase\Y7ec4\src\CloudBase.jl:124 [inlined]
     [28] #getObject#5
        @ C:\Users\mrufsvold\.julia\packages\CloudStore\6XBiA\src\s3.jl:30 [inlined]
     [29] (::CloudStore.API.var"#26#28"{})()
        @ CloudStore.API C:\Users\mrufsvold\.julia\packages\CloudStore\6XBiA\src\get.jl:228
Stacktrace:
 [1] sync_end(c::Channel{Any})
   @ Base .\task.jl:448
 [2] macro expansion
   @ .\task.jl:480 [inlined]
 [3] getObjectImpl(x::CloudBase.AWS.Bucket, key::String, out::Nothing; multipartThreshold::Int64, partSize::Int64, batchSize::Int64, allowMultipart::Bool, objectMaxSize::Nothing, decompress::Bool, zlibng::Bool, headers::Vector{…}, lograte::Bool, kw::@Kwargs{})
   @ CloudStore.API C:\Users\mrufsvold\.julia\packages\CloudStore\6XBiA\src\get.jl:214
 [4] getObjectImpl
   @ C:\Users\mrufsvold\.julia\packages\CloudStore\6XBiA\src\get.jl:98 [inlined]
 [5] get
   @ C:\Users\mrufsvold\.julia\packages\CloudStore\6XBiA\src\s3.jl:33 [inlined]
 [6] get
   @ C:\Users\mrufsvold\.julia\packages\CloudStore\6XBiA\src\CloudStore.jl:65 [inlined]
 [7] get(url::String, out::Nothing; region::String, nowarn::Bool, kw::@Kwargs{})
   @ CloudStore C:\Users\mrufsvold\.julia\packages\CloudStore\6XBiA\src\CloudStore.jl:78
 [8] get
   @ C:\Users\mrufsvold\.julia\packages\CloudStore\6XBiA\src\CloudStore.jl:76 [inlined]
 [9] top-level scope
   @ c:\Users\mrufsvold\Projects\DIL-price-transparency-psd\TableConsolidator.jl\playground.jl:116
Some type information was truncated. Use `show(err)` to see complete types. 

@quinnj
Copy link
Member

quinnj commented Aug 5, 2024

This sounds like some kind of version mismatch problem? Do you have branches checked out of HTTP/CloudStore/etc? The requests were working for me on the HTTP2 branch.

@mrufsvold
Copy link
Author

mrufsvold commented Aug 5, 2024

I switched back to the latest release of CloudStore.jl. I don't have HTTP.jl directly installed, so I had assumed that it would be resolved by CloudStore.jl's compat.

Here is my full environment:

julia> Pkg.status(;mode=PKGMODE_MANIFEST)
Project TableConsolidator v0.1.0
Status `C:\Users\mrufsvold\Projects\DIL-price-transparency-psd\TableConsolidator.jl\Manifest.toml`
  [fbe9abb3] AWS v1.91.0
  [1c724243] AWSS3 v0.11.2
  [1520ce14] AbstractTrees v0.4.5
  [7d9f7c33] Accessors v0.1.37
  [79e6a3ab] Adapt v4.0.4
  [dce04be8] ArgCheck v2.3.0
  [4c555306] ArrayLayouts v1.10.2
  [31f734f8] ArrowTypes v2.3.0
  [198e06fe] BangBang v0.4.3
  [9718e550] Baselet v0.1.1
  [d1d4a3ce] BitFlags v0.1.9
  [c3b6d118] BitIntegers v0.3.1
  [336ed68f] CSV v0.10.14
  [85eb1798] CloudBase v1.4.7
  [3365d9ee] CloudStore v1.6.4
  [5ba52731] CodecLz4 v0.4.4
  [944b1d66] CodecZlib v0.7.5
  [642d12eb] CodecZlibNG v0.1.0
  [6b39b394] CodecZstd v0.8.4
  [34da2185] Compat v4.15.0
  [a33af91c] CompositionsBase v0.1.2
  [f0e56b4a] ConcurrentUtilities v2.4.2
  [187b0558] ConstructionBase v1.5.6
  [9a962f9c] DataAPI v1.16.0
  [864edb3b] DataStructures v0.18.20
  [e2d170a0] DataValueInterfaces v1.0.0
  [55939f99] DecFP v1.3.2
  [244e2a9f] DefineSingletons v0.1.2
  [85a47980] Dictionaries v0.4.2
  [ffbed154] DocStringExtensions v0.9.3
  [460bff9d] ExceptionUnwrapping v0.1.10
  [e2ba6199] ExprTools v0.1.10
  [8f5d6c58] EzXML v1.2.0
  [5207ad80] FNVHash v0.1.0
  [937e9308] Figgy v1.1.1
  [48062228] FilePathsBase v0.9.21
  [1a297f60] FillArrays v1.11.0
  [069b7b12] FunctionWrappers v1.1.3
  [bc5e4493] GitHub v5.9.0
  [cd3eb016] HTTP v1.10.8
  [313cdc1a] Indexing v1.1.1
  [83e8ac13] IniFile v0.5.1
  [22cec73e] InitialValues v0.3.1
  [842dd82b] InlineStrings v1.4.2
  [3587e190] InverseFunctions v0.1.15
  [92d709cd] IrrationalConstants v0.2.2
  [c8e1da08] IterTools v1.10.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.5.0
  [682c06a0] JSON v0.21.4
  [0f8b85d8] JSON3 v1.14.0
⌅ [5078a376] LazyArrays v1.10.0
  [a4a7f996] LightBSON v0.2.20
  [2ab3a3ac] LogExpFunctions v0.3.28
  [e6f89c97] LoggingExtras v1.0.3
  [98105f81] LoggingFormats v1.5.0
  [6ac74813] MD5 v0.2.3
  [1914dd2f] MacroTools v0.5.13
⌅ [a3b82374] MatrixFactorizations v2.2.0
  [739be429] MbedTLS v1.1.9
  [128add7d] MicroCollections v0.2.0
⌅ [78c3b35d] Mocking v0.7.9
  [4d8831e6] OpenSSL v1.4.3
  [bac558e1] OrderedCollections v1.6.3
  [98572fba] Parquet2 v0.2.27
  [69de0a69] Parsers v2.8.1
  [2dfb63ee] PooledArrays v1.4.3
  [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [ae029012] Requires v1.3.0
  [20febd7b] Retry v0.4.1
  [91c51154] SentinelArrays v1.4.5
  [efcf1570] Setfield v1.1.1
  [777ac1f9] SimpleBufferStream v1.1.0
⌃ [59d4ed8c] Snappy v0.4.0
  [2133526b] SodiumSeal v0.1.1
  [276daf66] SpecialFunctions v2.4.0
  [03a91e81] SplitApplyCombine v1.2.3
  [171d559e] SplittablesBase v0.1.15
  [90137ffa] StaticArrays v1.9.7
  [1e83bf80] StaticArraysCore v1.4.3
  [856f2bd8] StructTypes v1.10.0
  [2da68c74] SymDict v0.3.0
  [ab02a1b2] TableOperations v1.2.0
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.12.0
  [9be31aac] Thrift2 v0.2.1
⌅ [3bb67fe8] TranscodingStreams v0.9.13
  [28d57a85] Transducers v0.4.82
  [9d95f2ec] TypedTables v1.4.6
  [5c2747f8] URIs v1.5.1
  [c4a57d5a] UnsafeArrays v1.0.6
  [ea10d353] WeakRefStrings v1.4.2
  [76eceee3] WorkerUtilities v1.6.1
  [228000da] XMLDict v0.4.1
  [47200ebd] DecFP_jll v2.0.3+1
  [dd4b983a] LZO_jll v2.10.2+0
  [94ce4f54] Libiconv_jll v1.17.0+0
  [5ced341a] Lz4_jll v1.10.0+0
  [a4b94fbf] NodeJS_16_jll v16.15.0+0
  [458c3c95] OpenSSL_jll v3.0.14+0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [02c8fc9c] XML2_jll v2.13.1+0
  [c62bbaca] ZlibNG_jll v2.1.6+0
  [3161d3a3] Zstd_jll v1.5.6+0
  [d59cea07] azurite_jll v3.29.0+0
  [a9144af2] libsodium_jll v1.0.20+0
  [ad83bbfc] minio_jll v1.0.0+1
  [fe1e1685] snappy_jll v1.2.0+0
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8ba89e20] Distributed
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.10.0
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [6462fe0b] Sockets
  [2f01184e] SparseArrays v1.10.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] LibCURL_jll v8.4.0+0
  [e37daf67] LibGit2_jll v1.6.4+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.2+1
  [14a3606d] MozillaCACerts_jll v2023.1.10
  [4536629a] OpenBLAS_jll v0.3.23+4
  [05823500] OpenLibm_jll v0.8.1+2
  [bea87d4a] SuiteSparse_jll v7.2.1+1
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.8.0+1
  [8e850ede] nghttp2_jll v1.52.0+1
  [3f19e933] p7zip_jll v17.4.0+2

@mrufsvold
Copy link
Author

Finding myself in a real mess of dependency issues, but the permission issue is solved. Gonna close this one and open a separate one to figure out what my environment ought to be so I don't clutter the history here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants