support logging

[cgwalters]

This is strongly related to #668

I think we should at least optionally support co-locating logs with our builds. As of today we don't really have a "configuration file" for cosa; could be an enviroment variable like env COSA_LOG=true or so?

And we'd need to bikeshed a bit about the log file layout; is there one buildlog.txt or is there logs/$artifact.txt etc. Today the RHCOS pipeline at least invokes cosa buildextend-$x concurrently so that argues for separate logs.

[ashcrow]

I'd advocate for separate logs. logs/$command-name.(log|txt) seem very reasonable to me. The standard python logger should be pretty easy to use for this.

import logging
import os.path


def create_logger(name, buildid):
    logger = logging.Logger(name, level=logging.INFO)
    out_path = os.path.join('logs', '{}.{}.txt'.format(name, buildid))
    handler = logging.FileHandler(out_path)
    handler.setFormatter(logging.Formatter(logging.BASIC_FORMAT))
    logger.handlers.append(handler)
    return logger

[cgwalters]

Sure, though not all of the code is Python. Seems a lot simpler and more robust to have cosa build-$x execute as a subprocess with |tee logs/build-$x.txt.

[ajeddeloh]

Ignoring implementation for a moment, we should log verbosely to files and less verbosely to stdout. Currently supermin and set -x print a lot of stuff that usually isn't too useful to stdout. I think there's still value in it, but it would be great to have that hidden away in files by default.

[ashcrow]

Sure, though not all of the code is Python. Seems a lot simpler and more robust to have cosa build-$x execute as a subprocess with |tee logs/build-$x.txt.

That would be simpler and more robust now. I really would like to see us use shell less, but the above would work well.

[jlebon]

Looking at this now, I think the old RHCOS pipeline did upload logs (encrypted), but it did it at the Jenkins level, which is probably where it makes more sense to do. The new pipeline doesn't yet have this ability but it could. I'd propose closing this in favour of coreos/fedora-coreos-pipeline#7 (and obviously cosa running in other runners than Jenkins either already have a logging solution or it can be added). Thoughts?