print() should be used to output information requested by a certain command, but not for logging the process to achieve it. log() should be used for the latter. The current code has the distinction not down clearly, fix that.
Since commit 02697af5, ExecContext.run() returns bytes for stdout and stderr and fixes that in calling code. The thing it did not fix was the code calling run_cmd(), which also made return bytes. This commit catches up on that.
ExecContext's .sudo() omits many of run()'s parameters, and this commit adds them. To avoid redundancy around repeating and massaging the long parameter list of both functions and their return values, it also adds some deeper changes:
- Make run(), _run(), sudo() and _sudo() always return instances of
Result. Before it was allowed to return a triplet of stdout,
stderr, and exit status.
- Have ExecContext stay out of the business of decoding the result
entirely. Result provides a convenience method .decode()
operating on stdout and stderr and leaves the decision to the
caller.
This entails miniscule adaptations in calling code, namely in
App.os_release, util.get_profile_env() and CmdListRepos._run().
- Wrap the _run() and _sudo() callbacks in a context manager object
of type CallContext to avoid code duplication.
- Consistently name the first argument to run(), _run(), sudo() and
_sudo() "cmd", not "args". The latter suggests that the caller is
omitting the executable, which is not the case.
Add the --verbose global option, which is made available as the App.verbose property.
Some functions still take a verbose parameter, but the type of these parameters is converted from bool to bool|None. The idea is that, if they are None, their verbosity falls back to the global default.
Allow to specify the ExecContext in a call to run_cmd(). This effectively makes run_cmd() an thin wrapper around ExecContext.run(), which is what's going to be used in the future. The wrapper is for backwards-compatibility.
The code below lib.distro, as left behind by the previous commit, is geared towards being directly used as a command-line API. This commit introduces the abstract base class Distro, a proxy for distribution-specific interactions. The proxy abstracts distro specifics into an API with proper method prototypes, not argparse.Namespace contents, and can thus be more easily driven by arbitrary code.
The Distro class is initialized with a member variable of type ExecContext, another new class introduced by this commit. It is designed to abstract the communication channel to the distribution instance. Currently only one specialization exists, Local, which interacts with the distribution and root file system it is running in, but is planned to be subclassed to support interaction via SSH, serial, chroot, or chains thereof.
jw-pkg distro dup got hung in a chroot environment. strace shows that write(2) into a pipe is the hanging syscall, with the write buffer hinting at zypper dup output.
I strongly suspect that run_cmd() tries to write stdout into the pipe which read_stream() fails to empty. So, make read_stream() more resilient by using read(4096) instead of readline(), which I suspect to be prone to hang on overlong lines.
run_cmd() with cmd_input == mode:interactive and verbose == true logs output too often. First, __log() is called, then pty.spawn() writes everything it reads from the PTY master to the terminal.
The fix it to not call __log() from _read() for the PTY reader.
Most run_xxx() return stdout and stderr. There's no way, really, for the caller to get hold of the exit code of the spawned executable. It can pass throw=true, catch, and assume a non-zero exit status. But that's not semantically clean, since the spawned function can well be a test function which is expected to return a non-zero status code, and the caller might be interested in what code that was, exactly.
The clearest way to solve this is to return the exit code as well. This commit does that.
This is a code maintenance commit: some run_xxx() helper functions take a string, some a list, and some just digest all arguments and pass them on as a list to exec() to be executed. That's highly inconsistent. This commit changes that to list-only.
Except for the run_cmd() method of SSHClient, which is still run as a shell method, because, erm, it's a shell. Might be changed in the future for consistency reasons.
Add a function get_profile_env(), a function returning environment variables from /etc/profile. Pass add=True to add its contents to the existing environment dictionary, overwriting old entries, or pass False to get the pristine content.
Add a parameter "output_encoding" to run_cmd(). The parameter allows the caller to specify if the output encoding should be detected as is by passing None (the default), if the output should be returned as undecoded bytes by passing the special string "bytes", or if the output should be treated as the encoding with the specified name and decoded to strings.
Don't log an Exception as {e} but as str(e) producing nicer output. Or as repr(e) if a backtrace is requested, because to people who can read backtraces, type info might be of interest. Also, remove pointless time stamps, those belong into the logging framework.
Move the body of BackendCmd.sudo() into a function. The rationale behind that is that its functionality is independent of the calling object for the most part, so having it in a function instead of a method is the more modular pattern.