Update dependency, fixes #8

[Thulium-Drake]

Update the dependency package for Mariadb.

Fixes #8

[dmsimard]

recheck

[dmsimard]

recheck

[dmsimard]

This is failing the mysql job on ubuntu-bionic with an error on the first ara_record task:

TASK [smoke-tests : Record data with no type] **********************************
task path: /home/zuul/.ara-tests/git/ara/tests/integration/roles/smoke-tests/tasks/ara-ops.yaml:30
The full traceback is:
Traceback (most recent call last):
  File "/home/zuul/.ara-tests/virtualenv/lib/python3.6/site-packages/ansible/executor/task_executor.py", line 147, in run
    res = self._execute()
  File "/home/zuul/.ara-tests/virtualenv/lib/python3.6/site-packages/ansible/executor/task_executor.py", line 630, in _execute
    self._handler = self._get_action_handler(connection=self._connection, templar=templar)
  File "/home/zuul/.ara-tests/virtualenv/lib/python3.6/site-packages/ansible/executor/task_executor.py", line 1082, in _get_action_handler
    collection_list=collections
  File "/home/zuul/.ara-tests/virtualenv/lib/python3.6/site-packages/ansible/plugins/loader.py", line 573, in get
    obj = obj(*args, **kwargs)
  File "/home/zuul/.ara-tests/virtualenv/lib/python3.6/site-packages/ara/plugins/action/ara_record.py", line 143, in __init__
    self.client = client_utils.active_client()
  File "/home/zuul/.ara-tests/virtualenv/lib/python3.6/site-packages/ara/clients/utils.py", line 50, in active_client
    return active_client._instance()
TypeError: 'NoneType' object is not callable
fatal: [localhost]: FAILED! => {
    "msg": "Unexpected failure during module execution.",
    "stdout": ""
}

I'm not sure why but it's worth investigating.

[Thulium-Drake]

Well, good news and bad news, this PR didn't broke it. But I have no clue what's going on...

I deployed a fresh ubuntu 18.04 lxc container. Installed ARA on it using the role (after prepping a mariadb server, we might need to update the docs about that thing with key lengths above 767 bytes, need to check)

So the role works, but this task does not:

- hosts: ara-ubt.lab
  tasks:
    - name: Record data with no type
      ara_record:
        key: "notype"
        value: "text"
      register: notype

But it doesn't matter whether the 'old' or 'new' dependency is installed (I also checked by dropping the ARA db a couple of times and it populates just fine with the 'new' dependency)

Bad news, no clue what this is. It is Ubuntu 18.04 related though, I also ran the faulty task on a Debian buster controller, no problems (same ARA target btw)

[dmsimard]

Thanks for looking at this !

I found out that there are different images of Ubuntu used in our Zuul jobs right now -- depending if the job runs in AWS or an OpenStack cloud. This explains some of the odd flapping and regressions we've seen but hey, we got some bug fixes out of it :)

I haven't seen this reproduce in other PRs and they merged successfully -- I'll ask for a recheck here and I'll try to reproduce the issue locally troubleshoot.

[dmsimard]

recheck

[Thulium-Drake]

Hi @dmsimard,

Any other avenue we can pursue? As mentioned in #8 , even when I remove all mariadb related packages (on the Ansible node reporting to ARA, not the ARA node itself, which makes sense as the ARA client is HTTP based, right? :-) ) I still can't fix the issue.

I also removed them from my testing system that does run ARA, and it still works fine

As the pacakge is not required (of what I was able to test), I have updated the PR to remove it altogether.

[dmsimard]

@Thulium-Drake now the job is failing when installing mysqlclient from pip ¯\(ツ)/¯

It determines that those are the missing packages:

Then installs them:

The failure is in the mysql backend job and so django requires mysqlclient to be installed -- it's a server dependency, not a client one.. in other words, it's required for ara-manage commands (such as migrate) but not for ara playbook list (unless using the offline client).

Anyway, I'd expect the mariadb-client package to provide the necessary client binaries to install mysqlclient from pip but maybe they come from the devel package ? Googling seems to suggest the devel package is necessary: https://stackoverflow.com/questions/54350881/error-installing-mysqlclient-for-python-on-ubuntu-18-04

[Thulium-Drake]

Fair enough, makes sense, but I really don't understand why this breaks on Ubuntu 18.. Is there any way how I can test this on Ubuntu 20 with the CI?

Maybe this particular transitional package is borked on 18?

[dmsimard]

Is there any way how I can test this on Ubuntu 20 with the CI?

Our Zuul's configuration shows we have currently have Xenial and Bionic.

Looking at the OpenStack configuration (another zuul deployment) shows there is a Focal image so it is implemented but we don't have it yet.

We can use the addition of a fedora 33 image recently as an example: ansible-network/windmill-config@ed054b3#diff-6c807ed8fd39d8131387aba05480ec2b29da62c8b39f803d6a4ad59d9cf42578

Edit: on a second read of that addition, that's AWS only since it runs off of the Fedora AMI which works but I must check if we'd rather do it on OpenStack instead which has a different implementation using images built with diskimage builder.

[Thulium-Drake]

recheck

[Thulium-Drake]

Time to have another look at this one :-)

[dmsimard]

The job failure is unrelated, it's the same issue as I pointed out in #36 (comment)

This failure is on ubuntu bionic again. I'll try to find out why it's happening but it seems intermittent ?

[dmsimard]

recheck

[dmsimard]

I created an issue for the ara_record traceback: ansible-community/ara#325

[dmsimard]

recheck

[dmsimard]

recheck

[dmsimard]

I requested the CI nodes to be held if the job fails again so I can introspect a bit.

[dmsimard]

libmariadb-dev is super old in 18.04, see my investigation here: ansible-community/ara#325 (comment)

libmariadbclient-dev is the right library to use and works in 20.04 as well.

With that in mind, maybe we should leave things as-is and revisit if it does break or if we have reports of issues.

[Thulium-Drake]

I agree, shall we also close #8 in favor of not breaking it? I've noticed in Debian that the package is being redirected properly too.

Maybe revisit things in a few years (when they actually remove libmariadbclient-dev from the repos) :-)

[dmsimard]

I agree, shall we also close #8 in favor of not breaking it? I've noticed in Debian that the package is being redirected properly too.

Maybe revisit things in a few years (when they actually remove libmariadbclient-dev from the repos) :-)

Works for me.

[Thulium-Drake]

It broke again, I recently upgraded my system to Debian 11, and it now runs into this issue as the libmariadbclient-dev package is not in the repos anymore.

However, we did settle a few months ago that this breaks a lot of stuff in the pipeline.

I did some checking, the last version of Ubuntu to ship this package is 20.04 LTS. So, I think we really need to figure out a solution now :)

[Thulium-Drake]

So I made a patch that will default to the new package name for newer distro's, except for Ubuntu 18, as it breaks with that new package. This should fix it for most deployments :-)

[Thulium-Drake]

recheck

[Thulium-Drake]

The tests succeeded! Only some timeouts and an unknown failure for the nginx test

[Thulium-Drake]

@dmsimard Ping! :-)

[softwarefactory-project-zuul]

Build succeeded.

✔️ ara-role-api-distributed-sqlite SUCCESS in 13m 43s
✔️ ara-role-api-mysql SUCCESS in 13m 01s
✔️ ara-role-api-postgresql SUCCESS in 12m 19s
✔️ ara-role-api-gunicorn-nginx SUCCESS in 10m 08s
✔️ ara-role-api-fedora-packages SUCCESS in 5m 41s (non-voting)

[dmsimard]

Hmm, the problem is that we no longer have ubuntu/debian coverage at this time.
Let me try and make either available in CI so we can validate it.