Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add option to disable cookie persistence in Client/AsyncClient #2992

Open
fastily opened this issue Dec 8, 2023 · 3 comments · May be fixed by #3065
Open

Add option to disable cookie persistence in Client/AsyncClient #2992

fastily opened this issue Dec 8, 2023 · 3 comments · May be fixed by #3065
Labels
enhancement New feature or request

Comments

@fastily
Copy link

fastily commented Dec 8, 2023

Initially raised as discussion #1533

I've been using httpx's AsyncClient as a web spider, however I have noticed that cookies are automatically persisted and there's no easy way to disable this. My workaround has been to subclass python's http.cookiejar.CookieJar like so:

from http.cookiejar import CookieJar

class NullCookieJar(CookieJar):
    """A CookieJar that rejects all cookies."""

    def extract_cookies(self, *_):
        """For extracting and saving cookies.  This implementation does nothing"""
        pass

    def set_cookie(self, _):
        """Normally for setting a cookie.  This implementation does nothing"""
        pass

Would it be possible to:

  1. add an option to disable cookie persistence to Client/AsyncClient or
  2. include this implementation of NullCookieJar in httpx as a utility class?

Thanks!
@karpetrosyan
Copy link
Member

Hi!
Perhaps you could remove the Set-Cookie header before it reaches the client instance?

Example:

import httpx


class DisableCookieTransport(httpx.BaseTransport):
    def __init__(self, transport: httpx.BaseTransport):
        self.transport = transport

    def handle_request(self, request: httpx.Request) -> httpx.Response:
        response = self.transport.handle_request(request)
        del response.headers["set-cookie"]
        return response


client = httpx.Client(transport=DisableCookieTransport(httpx.HTTPTransport()))

response = client.get("https://httpbin.org/cookies/set?foo=bar")

@MarkWine MarkWine linked a pull request Jan 17, 2024 that will close this issue
Open
3 tasks
@idrissbellil
Copy link

idrissbellil commented May 27, 2024 •
edited
Loading

For AsyncClient I am guessing the workaround is:

class AsyncDisableCookieTransport(httpx.AsyncBaseTransport):
    def __init__(self, transport: httpx.AsyncBaseTransport):
        self.transport = transport

    async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
        response = await self.transport.handle_async_request(request)
        response.headers.pop("set-cookie", None)
        return response

client = httpx.AsyncClient(transport=AsyncDisableCookieTransport(httpx.AsyncHTTPTransport()))

@tomchristie
Copy link
Member

Perhaps this is a common enough use-case that we should add cookies = httpx.NoCookies(), in line with @fastily's suggestion?

@fantix fantix mentioned this issue Sep 12, 2024
Merged
4 tasks
@tomchristie tomchristie added the enhancement New feature or request label Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add persistent cookies option to Client MarkWine/httpx
4 participants
@tomchristie @fastily @idrissbellil @karpetrosyan