Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

django.db.utils.InterfaceError: connection already closed when running pytest #1099

Open
k0286 opened this issue Dec 6, 2023 · 1 comment

Comments

@k0286
Copy link

k0286 commented Dec 6, 2023

environment:

Postgresql 14.7.0
Python 3.10
Django==4.2.8
djangorestframework==3.14.0
pytest-django==4.7.0
pytest==7.4.3
pytest-celery==0.0.0
pytest-django==4.7.0
pytest-elasticsearch==4.0.2
pytest-factoryboy==2.6.0
pytest-mock==3.12.0
pytest-xdist==3.5.0

the error log

self = <django.contrib.sessions.backends.db.SessionStore object at 0x7f28520365f0>
no_load = False

    def _get_session(self, no_load=False):
        """
        Lazily load session from storage (unless "no_load" is True, when only
        an empty dict is stored) and store it in the current instance.
        """
        self.accessed = True
        try:
>           return self._session_cache
E           AttributeError: 'SessionStore' object has no attribute '_session_cache'

/usr/local/lib/python3.10/site-packages/django/contrib/sessions/backends/base.py:187: AttributeError

During handling of the above exception, another exception occurred:

self = <DatabaseWrapper vendor='postgresql' alias='default'>, name = None

    def _cursor(self, name=None):
        self.close_if_health_check_failed()
        self.ensure_connection()
        with self.wrap_database_errors:
>           return self._prepare_cursor(self.create_cursor(name))

/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py:308: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py:26: in inner
    return func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DatabaseWrapper vendor='postgresql' alias='default'>, name = None

    @async_unsafe
    def create_cursor(self, name=None):
        if name:
            # In autocommit mode, the cursor will be used outside of a
            # transaction, hence use a holdable cursor.
            cursor = self.connection.cursor(
                name, scrollable=False, withhold=self.connection.autocommit
            )
        else:
>           cursor = self.connection.cursor()
E           psycopg2.InterfaceError: connection already closed

/usr/local/lib/python3.10/site-packages/django/db/backends/postgresql/base.py:330: InterfaceError

The above exception was the direct cause of the following exception:

client = <django.test.client.Client object at 0x7f28b0a61c30>
django_user_model = <class 'django.contrib.auth.models.User'>

    @pytest.mark.django_db
    def test_auth_view(client, django_user_model):
        user = django_user_model.objects.create_user("test", "test1234", is_staff=True, is_superuser=True)
        client.force_login(user)
        url = reverse('admin:auth_user_add')
>       resp = client.post(url, data={"username": "test2", "password1": "1qaz@WSX3edc", "password2": "1qaz@WSX3edc"}, follow=True)

notification/tests/test_admin.py:63: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.10/site-packages/django/test/client.py:957: in post
    response = self._handle_redirects(
/usr/local/lib/python3.10/site-packages/django/test/client.py:1173: in _handle_redirects
    response = request_method(
/usr/local/lib/python3.10/site-packages/django/test/client.py:927: in get
    response = super().get(path, data=data, secure=secure, headers=headers, **extra)
/usr/local/lib/python3.10/site-packages/django/test/client.py:457: in get
    return self.generic(
/usr/local/lib/python3.10/site-packages/django/test/client.py:609: in generic
    return self.request(**r)
/usr/local/lib/python3.10/site-packages/django/test/client.py:891: in request
    self.check_exception(response)
/usr/local/lib/python3.10/site-packages/django/test/client.py:738: in check_exception
    raise exc_value
/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py:55: in inner
    response = get_response(request)
/usr/local/lib/python3.10/site-packages/django/utils/deprecation.py:136: in __call__
    response = self.process_response(request, response)
activity/middleware.py:44: in process_response
    user_id = request.user.id
/usr/local/lib/python3.10/site-packages/django/utils/functional.py:266: in inner
    self._setup()
/usr/local/lib/python3.10/site-packages/django/utils/functional.py:419: in _setup
    self._wrapped = self._setupfunc()
/usr/local/lib/python3.10/site-packages/django/contrib/auth/middleware.py:25: in <lambda>
    request.user = SimpleLazyObject(lambda: get_user(request))
/usr/local/lib/python3.10/site-packages/django/contrib/auth/middleware.py:11: in get_user
    request._cached_user = auth.get_user(request)
/usr/local/lib/python3.10/site-packages/django/contrib/auth/__init__.py:191: in get_user
    user_id = _get_user_session_key(request)
/usr/local/lib/python3.10/site-packages/django/contrib/auth/__init__.py:60: in _get_user_session_key
    return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
/usr/local/lib/python3.10/site-packages/django/contrib/sessions/backends/base.py:53: in __getitem__
    return self._session[key]
/usr/local/lib/python3.10/site-packages/django/contrib/sessions/backends/base.py:192: in _get_session
    self._session_cache = self.load()
/usr/local/lib/python3.10/site-packages/django/contrib/sessions/backends/db.py:42: in load
    s = self._get_session_from_db()
/usr/local/lib/python3.10/site-packages/django/contrib/sessions/backends/db.py:32: in _get_session_from_db
    return self.model.objects.get(
/usr/local/lib/python3.10/site-packages/django/db/models/manager.py:87: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:633: in get
    num = len(clone)
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:380: in __len__
    self._fetch_all()
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:1881: in _fetch_all
    self._result_cache = list(self._iterable_class(self))
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:91: in __iter__
    results = compiler.execute_sql(
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:1560: in execute_sql
    cursor = self.connection.cursor()
/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py:26: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py:330: in cursor
    return self._cursor()
/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py:307: in _cursor
    with self.wrap_database_errors:
/usr/local/lib/python3.10/site-packages/django/db/utils.py:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py:308: in _cursor
    return self._prepare_cursor(self.create_cursor(name))
/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py:26: in inner
    return func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DatabaseWrapper vendor='postgresql' alias='default'>, name = None

    @async_unsafe
    def create_cursor(self, name=None):
        if name:
            # In autocommit mode, the cursor will be used outside of a
            # transaction, hence use a holdable cursor.
            cursor = self.connection.cursor(
                name, scrollable=False, withhold=self.connection.autocommit
            )
        else:
>           cursor = self.connection.cursor()
E           django.db.utils.InterfaceError: connection already closed

/usr/local/lib/python3.10/site-packages/django/db/backends/postgresql/base.py:330: InterfaceError

the test case

@pytest.mark.django_db
def test_auth_view(client, django_user_model):
    user = django_user_model.objects.create_user(
        "test", "test1234", is_staff=True, is_superuser=True
    )
    client.force_login(user)
    url = reverse("admin:auth_user_add")
    resp = client.post(
        url,
        data={"username": "test2", "password1": "1qaz@WSX3edc", "password2": "1qaz@WSX3edc"},
        follow=True,
    )
    assert resp.status_code == 200
    assert User.objects.filter(username="test2").exists()

It seems like the connection is close by some one, probably client, before the function is over.
the test case can pass if I use the @pytest.mark.django_db(transaction=True) marker. But I don't know the reason.

@hvieira
Copy link

hvieira commented Oct 1, 2024

I was having a similar issue, but with (I assume) a significant difference of not using @async_unsafe.

In my case, I was using a fixture to create a requests client - https://www.django-rest-framework.org/api-guide/testing/#requestsclient. After some experiments, I figured out that using this client instead of the usual django client resulted in connection already closed errors - perhaps the testing django client is more closely tied to the test flow?

environment info:
python 3.10.13
pytest-django - 4.9.0
pytest - 8.3.3
django - 5.1.1
psycopg2 - 2.9.9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants