guestfs-hacking (1) Linux Manual Page

NAME

guestfs-hacking – extending and contributing to libguestfs

DESCRIPTION

This manual page is for hackers who want to extend libguestfs itself.

THE SOURCE CODE

Libguestfs source is located in the github repository https://github.com/libguestfs/libguestfs

Large amounts of boilerplate code in libguestfs (RPC, bindings, documentation) are generated. This means that many source files will appear to be missing from a straightforward git checkout. You have to run the generator ("./autogen.sh && make -C generator") in order to create those files.

Libguestfs uses an autotools-based build system, with the main files being configure.ac and Makefile.am. See “THE BUILD SYSTEM”.

The generator subdirectory contains the generator, plus files describing the API. The lib subdirectory contains source for the library. The appliance and daemon subdirectories contain the source for the code that builds the appliance, and the code that runs in the appliance respectively. Other directories are covered in the section “SOURCE CODE SUBDIRECTORIES” below.

Apart from the fact that all API entry points go via some generated code, the library is straightforward. (In fact, even the generated code is designed to be readable, and should be read as ordinary code). Some actions run entirely in the library, and are written as C functions in files under lib. Others are forwarded to the daemon where (after some generated RPC marshalling) they appear as C functions in files under daemon.

To build from source, first read the guestfs-building(1).

SOURCE CODE SUBDIRECTORIES

There are a lot of subdirectories in the source tree! Which ones should you concentrate on first? lib and daemon which contain the source code of the core library. generator is the code generator described above, so that is important. The Makefile.am in the root directory will tell you in which order the subdirectories get built. And then if you are looking at a particular tool (eg. v2v) or language binding (eg. python), go straight to that subdirectory, but remember that if you didn’t run the generator yet, then you may find files which appear to be missing.

align

virt-alignment-scan(1) command and documentation.

appliance

The libguestfs appliance, build scripts and so on.

bash

Bash tab-completion scripts.

build-aux

Various build scripts used by autotools.

builder

virt-builder(1) command and documentation.

cat

The virt-cat(1), virt-filesystems(1), virt-log(1), virt-ls(1) and virt-tail(1) commands and documentation.

common

Various libraries of internal code can be found in the common subdirectory:

common/edit

Common code for interactively and non-interactively editing files within a libguestfs filesystem.

common/errnostring

The communication protocol used between the library and the daemon running inside the appliance has to encode errnos as strings, which is handled by this library.

common/miniexpect

A copy of the miniexpect library from http://git.annexia.org/?p=miniexpect.git;a=summary. This is used in virt-p2v.

common/mlaugeas

Bindings for the Augeas library. These come from the ocaml-augeas library http://git.annexia.org/?p=ocaml-augeas.git

common/mlgettext

Small, generated wrapper which allows libguestfs to be compiled with or without ocaml-gettext. This is generated by ./configure.

common/mlpcre

Lightweight OCaml bindings for Perl Compatible Regular Expressions (PCRE). Note this is not related in any way to Markus Mottl’s ocaml-pcre library.

common/mlprogress

OCaml bindings for the progress bar functions (see common/progress).

common/mlstdutils

A library of pure OCaml utility functions used in many places.

common/mltools

OCaml utility functions only used by the OCaml virt tools (like "virt-sysprep", "virt-v2v" etc.)

common/mlutils

OCaml bindings for C functions in "common/utils", and some POSIX bindings which are missing from the OCaml stdlib.

common/mlvisit

OCaml bindings for the visit functions (see common/visit).

common/mlxml

OCaml bindings for the libxml2 library.

common/options

Common options parsing for guestfish, guestmount and some virt tools.

common/parallel

A framework used for processing multiple libvirt domains in parallel.

common/progress

Common code for printing progress bars.

common/protocol

The XDR-based communication protocol used between the library and the daemon running inside the appliance is defined here.

common/qemuopts

Mini-library for writing qemu command lines and qemu config files.

common/structs

Common code for printing and freeing libguestfs structs, used by the library and some tools.

common/utils

Various utility functions used throughout the library and tools.

common/visit

Recursively visit a guestfs filesystem hierarchy.

common/windows

Utility functions for handling Windows drive letters.

contrib

Outside contributions, experimental parts.

customize

virt-customize(1) command and documentation.

daemon

The daemon that runs inside the libguestfs appliance and carries out actions.

df

virt-df(1) command and documentation.

dib

virt-dib(1) command and documentation.

diff

virt-diff(1) command and documentation.

docs

Miscellaneous manual pages.

edit

virt-edit(1) command and documentation.

examples

C API example code.

fish

guestfish(1), the command-line shell, and various shell scripts built on top such as virt-copy-in(1), virt-copy-out(1), virt-tar-in(1), virt-tar-out(1).

format

virt-format(1) command and documentation.

fuse

guestmount(1), FUSE (userspace filesystem) built on top of libguestfs.

generator

The crucially important generator, used to automatically generate large amounts of boilerplate C code for things like RPC and bindings.

get-kernel

virt-get-kernel(1) command and documentation.

gnulib

Gnulib is used as a portability library. A copy of gnulib is included under here.

inspector

virt-inspector(1), the virtual machine image inspector.

lib

Source code to the C library.

logo

Logo used on the website. The fish is called Arthur by the way.

m4

M4 macros used by autoconf. See “THE BUILD SYSTEM”.

make-fs

virt-make-fs(1) command and documentation.

p2v

virt-p2v(1) command, documentation and scripts for building the virt-p2v ISO or disk image.

po

Translations of simple gettext strings.

po-docs

The build infrastructure and PO files for translations of manpages and POD files. Eventually this will be combined with the po directory, but that is rather complicated.

rescue

virt-rescue(1) command and documentation.

resize

virt-resize(1) command and documentation.

sparsify

virt-sparsify(1) command and documentation.

sysprep

virt-sysprep(1) command and documentation.

tests

Tests.

test-data

Files and other test data used by the tests.

test-tool

Test tool for end users to test if their qemu/kernel combination will work with libguestfs.

tmp

Used for temporary files when running the tests (instead of /tmp etc). The reason is so that you can run multiple parallel tests of libguestfs without having one set of tests overwriting the appliance created by another.

tools

Command line tools written in Perl (virt-win-reg(1) and many others).

utils

Miscellaneous utilities, such as "boot-benchmark".

v2v

virt-v2v(1) command and documentation.

website

The http://libguestfs.org website files.

csharp

erlang

gobject

golang

haskell

java

lua

ocaml

php

perl

python

ruby

Language bindings.

THE BUILD SYSTEM

Libguestfs uses the GNU autotools build system (autoconf, automake, libtool).

The ./configure script is generated from configure.ac and m4/guestfs-*.m4. Most of the configure script is split over many m4 macro files by topic, for example m4/guestfs-daemon.m4 deals with the dependencies of the daemon.

The job of the top level Makefile.am is mainly to list the subdirectories ("SUBDIRS") in the order they should be compiled.

common-rules.mk is included in every Makefile.am (top level and subdirectories). subdir-rules.mk is included only in subdirectory Makefile.am files.

There are many make targets. Use this command to list them all:

 make help

EXTENDING LIBGUESTFS

ADDING A NEW API

Because large amounts of boilerplate code in libguestfs are generated, this makes it easy to extend the libguestfs API.

To add a new API action there are two changes:

1.

You need to add a description of the call (name, parameters, return type, tests, documentation) to generator/actions_*.ml and possibly generator/proc_nr.ml.

There are two sorts of API action, depending on whether the call goes through to the daemon in the appliance, or is serviced entirely by the library (see “ARCHITECTURE” in guestfs-internals(1)). “guestfs_sync” in guestfs(3) is an example of the former, since the sync is done in the appliance. “guestfs_set_trace” in guestfs(3) is an example of the latter, since a trace flag is maintained in the handle and all tracing is done on the library side.

Most new actions are of the first type, and get added to the "daemon_functions" list. Each function has a unique procedure number used in the RPC protocol which is assigned to that action when we publish libguestfs and cannot be reused. Take the latest procedure number and increment it.

For library-only actions of the second type, add to the "non_daemon_functions" list. Since these functions are serviced by the library and do not travel over the RPC mechanism to the daemon, these functions do not need a procedure number, and so the procedure number is set to "-1".

2.

Implement the action (in C):

For daemon actions, implement the function "do_<name>" in the "daemon/" directory.

For library actions, implement the function "guestfs_impl_<name>" in the "lib/" directory.

In either case, use another function as an example of what to do.

3.

As an alternative to step 2: Since libguestfs 1.38, daemon actions can be implemented in OCaml. You have to set the "impl = OCaml ..." flag in the generator. Take a look at daemon/file.ml for an example.

After making these changes, use "make" to compile.

Note that you don’t need to implement the RPC, language bindings, manual pages or anything else. It’s all automatically generated from the OCaml description.

Adding tests for an API

You can supply zero or as many tests as you want per API call. The tests can either be added as part of the API description (generator/actions_*.ml), or in some rarer cases you may want to drop a script into "tests/*/". Note that adding a script to "tests/*/" is slower, so if possible use the first method.

The following describes the test environment used when you add an API test in actions_*.ml.

The test environment has 4 block devices:

/dev/sda 2 GB

General block device for testing.

/dev/sdb 2 GB

/dev/sdb1 is an ext2 filesystem used for testing filesystem write operations.

/dev/sdc 10 MB

Used in a few tests where two block devices are needed.

/dev/sdd

ISO with fixed content (see images/test.iso).

To be able to run the tests in a reasonable amount of time, the libguestfs appliance and block devices are reused between tests. So don’t try testing “guestfs_kill_subprocess” in guestfs(3) :-x

Each test starts with an initial scenario, selected using one of the "Init*" expressions, described in generator/types.ml. These initialize the disks mentioned above in a particular way as documented in types.ml. You should not assume anything about the previous contents of other disks that are not initialized.

You can add a prerequisite clause to any individual test. This is a run-time check, which, if it fails, causes the test to be skipped. Useful if testing a command which might not work on all variations of libguestfs builds. A test that has prerequisite of "Always" means to run unconditionally.

In addition, packagers can skip individual tests by setting environment variables before running "make check".

 SKIP_TEST_<CMD>_<NUM>=1

eg: "SKIP_TEST_COMMAND_3=1" skips test #3 of “guestfs_command” in guestfs(3).

or:

 SKIP_TEST_<CMD>=1

eg: "SKIP_TEST_ZEROFREE=1" skips all “guestfs_zerofree” in guestfs(3) tests.

Packagers can run only certain tests by setting for example:

 TEST_ONLY="vfs_type zerofree"

See tests/c-api/tests.c for more details of how these environment variables work.

Debugging new APIs

Test new actions work before submitting them.

You can use guestfish to try out new commands.

Debugging the daemon is a problem because it runs inside a minimal environment. However you can fprintf messages in the daemon to stderr, and they will show up if you use "guestfish -v".

ADDING A NEW LANGUAGE BINDING

All language bindings must be generated by the generator (see the generator subdirectory).

There is no documentation for this yet. We suggest you look at an existing binding, eg. generator/ocaml.ml or generator/perl.ml.

Adding tests for language bindings

Language bindings should come with tests. Previously testing of language bindings was rather ad-hoc, but we have been trying to formalize the set of tests that every language binding should use.

Currently only the OCaml and Perl bindings actually implement the full set of tests, and the OCaml bindings are canonical, so you should emulate what the OCaml tests do.

This is the numbering scheme used by the tests:

 - 000+ basic tests:
 
   010  load the library
   020  create
   030  create-flags
   040  create multiple handles
   050  test setting and getting config properties
   060  explicit close
   065  implicit close (in GC'd languages)
   070  optargs
   080  version
   090  retvalues
 
 - 100  launch, create partitions and LVs and filesystems
 
 - 400+ events:
 
   410  close event
   420  log messages
   430  progress messages
 
 - 800+ regression tests (specific to the language)
 
 - 900+ any other custom tests for the language

To save time when running the tests, only 100, 430, 800+, 900+ should launch the handle.

FORMATTING CODE

Our C source code generally adheres to some basic code-formatting conventions. The existing code base is not totally consistent on this front, but we do prefer that contributed code be formatted similarly. In short, use spaces-not-TABs for indentation, use 2 spaces for each indentation level, and other than that, follow the K&R style.

If you use Emacs, add the following to one of your start-up files (e.g., ~/.emacs), to help ensure that you get indentation right:

;
;
;
In libguestfs, indent with spaces everywhere(not TABs).;
;
; Exceptions: Makefile and ChangeLog modes.
 (add-hook 'find-file-hook
     '(lambda () (if (and buffer-file-name
                          (string-match "/libguestfs\>"
                              (buffer-file-name))
                          (not (string-equal mode-name "Change Log"))
                          (not (string-equal mode-name "Makefile")))
                     (setq indent-tabs-mode nil))))
 
 ;
;
; When editing C sources in libguestfs, use this style.
 (defun libguestfs-c-mode ()
   "C mode with adjusted defaults for use with libguestfs."
   (interactive)
   (c-set-style "K&R")
   (setq c-indent-level 2)
   (setq c-basic-offset 2))
 (add-hook 'c-mode-hook
           '(lambda () (if (string-match "/libguestfs\>"
                               (buffer-file-name))
                           (libguestfs-c-mode))))

TESTING YOUR CHANGES

Turn warnings into errors when developing to make warnings hard to ignore:

 ./configure --enable-werror

Useful targets are:

"make check"

Runs the regular test suite.

This is implemented using the regular automake "TESTS" target. See the automake documentation for details.

"make check-valgrind"

Runs a subset of the test suite under valgrind.

See “VALGRIND” below.

"make check-valgrind-local-guests"

Runs a subset of the test suite under valgrind using locally installed libvirt guests (read-only).

"make check-direct"

Runs all tests using default appliance back-end. This only has any effect if a non-default backend was selected using "./configure --with-default-backend=..."

"make check-valgrind-direct"

Run a subset of the test suite under valgrind using the default appliance back-end.

"make check-uml"

Runs all tests using the User-Mode Linux backend.

As there is no standard location for the User-Mode Linux kernel, you have to set "LIBGUESTFS_HV" to point to the kernel image, eg:

 make check-uml LIBGUESTFS_HV=~/d/linux-um/vmlinux

"make check-valgrind-uml"

Runs all tests using the User-Mode Linux backend, under valgrind.

As above, you have to set "LIBGUESTFS_HV" to point to the kernel.

"make check-with-upstream-qemu"

Runs all tests using a local qemu binary. It looks for the qemu binary in QEMUDIR (defaults to $HOME/d/qemu), but you can set this to another directory on the command line, eg:

 make check-with-upstream-qemu QEMUDIR=/usr/src/qemu

"make check-with-upstream-libvirt"

Runs all tests using a local libvirt. This only has any effect if the libvirt backend was selected using "./configure --with-default-backend=libvirt"

It looks for libvirt in LIBVIRTDIR (defaults to $HOME/d/libvirt), but you can set this to another directory on the command line, eg:

 make check-with-upstream-libvirt LIBVIRTDIR=/usr/src/libvirt

"make check-slow"

Runs some slow/long-running tests which are not run by default.

To mark a test as slow/long-running:

•

Add it to the list of "TESTS" in the Makefile.am, just like a normal test.

•

Modify the test so it checks if the "SLOW=1" environment variable is set, and if not set it skips (ie. returns with exit code 77). If using $TEST_FUNCTIONS, you can call the function "slow_test" for this.

•

Add a variable "SLOW_TESTS" to the Makefile.am listing the slow tests.

•

Add a rule to the Makefile.am:

 check-slow:
   $(MAKE) check TESTS="$(SLOW_TESTS)" SLOW=1

"sudo make check-root"

Runs some tests which require root privileges. These are supposed to be safe, but take care. You have to run this as root (eg. using sudo(8) explicitly).

To mark a test as requiring root:

•

Add it to the list of "TESTS" in the Makefile.am, just like a normal test.

•

Modify the test so it checks if euid == 0, and if not set it skips (ie. returns with exit code 77). If using $TEST_FUNCTIONS, you can call the function "root_test" for this.

•

Add a variable "ROOT_TESTS" to the Makefile.am listing the root tests.

•

Add a rule to the Makefile.am:

 check-root:
   $(MAKE) check TESTS="$(ROOT_TESTS)"

"make check-all"

Equivalent to running all "make check*" rules except "check-root".

"make check-release"

Runs a subset of "make check*" rules that are required to pass before a tarball can be released. Currently this is:

•

check

•

check-valgrind

•

check-direct

•

check-valgrind-direct

•

check-slow

"make installcheck"

Run "make check" on the installed copy of libguestfs.

The version of installed libguestfs being tested, and the version of the libguestfs source tree must be the same.

Do:

 ./autogen.sh
 make clean ||:
 make
 make installcheck

VALGRIND

When you do "make check-valgrind", it searches for any Makefile.am in the tree that has a "check-valgrind:" target and runs it.

Writing the Makefile.am and tests correctly to use valgrind and working with automake parallel tests is subtle.

If your tests are run via a shell script wrapper, then in the wrapper use:

 $VG virt-foo

and in the Makefile.am use:

 check-valgrind:
     make VG="@VG@" check

However, if your binaries run directly from the "TESTS" rule, you have to modify the Makefile.am like this:

 LOG_COMPILER = $(VG)
 
 check-valgrind:
     make VG="@VG@" check

In either case, check that the right program is being tested by examining the tmp/valgrind* log files carefully.

SUBMITTING PATCHES

Submit patches to the mailing list: http://www.redhat.com/mailman/listinfo/libguestfs and CC to rjones [at] redhat.com.

You do not need to subscribe to the mailing list if you don’t want to. There may be a short delay while your message is moderated.

DAEMON CUSTOM PRINTF FORMATTERS

In the daemon code we have created custom printf formatters %Q and %R, which are used to do shell quoting.

%Q

Simple shell quoted string. Any spaces or other shell characters are escaped for you.

%R

Same as %Q except the string is treated as a path which is prefixed by the sysroot.

For example:

 asprintf (&cmd, "cat %R", path);

would produce "cat /sysroot/some\ path\ with\ spaces"

Note: Do not use these when you are passing parameters to the "command{,r,v,rv}()" functions. These parameters do NOT need to be quoted because they are not passed via the shell (instead, straight to exec). You probably want to use the "sysroot_path()" function however.

INTERNATIONALIZATION (I18N) SUPPORT

We support i18n (gettext anyhow) in the library.

However many messages come from the daemon, and we don’t translate those at the moment. One reason is that the appliance generally has all locale files removed from it, because they take up a lot of space. So we’d have to readd some of those, as well as copying our PO files into the appliance.

Debugging messages are never translated, since they are intended for the programmers.

MISCELLANEOUS TOPICS

HOW OCAML PROGRAMS ARE COMPILED AND LINKED

Mostly this section is “how we make automake & ocamlopt work together” since OCaml programs themselves are easy to compile.

Automake has no native support for OCaml programs, ocamlc nor ocamlopt. What we do instead is to treat OCaml programs as C programs which happen to contain these “other objects” ("DEPENDENCIES" in automake-speak) that happen to be the OCaml objects. This works because OCaml programs usually have C files for native bindings etc.

So a typical program is described as just its C sources:

 virt_v2v_SOURCES = ... utils-c.c xml-c.c

For programs that have no explicit C sources, we create an empty dummy.c file, and list that instead:

 virt_resize_SOURCES = dummy.c

The OCaml objects which contain most of the code are listed as automake dependencies (other dependencies may also be listed):

 virt_v2v_DEPENDENCIES = ... cmdline.cmx v2v.cmx

The only other special thing we need to do is to provide a custom link command. This is needed because automake won’t assemble the ocamlopt command, the list of objects and the "-cclib" libraries in the correct order otherwise.

 virt_v2v_LINK = \
     $(top_srcdir)/ocaml-link.sh -cclib '-lutils -lgnu' -- ...

The actual rules, which you can examine in v2v/Makefile.am, are a little bit more complicated than this because they have to handle:

•

Compiling for byte code or native code.

•

The pattern rules needed to compile the OCaml sources to objects.

These are now kept in subdir-rules.mk at the top level, which is included in every subdirectory Makefile.am.

•

Adding OCaml sources files to "EXTRA_DIST".

Automake isn’t aware of the complete list of sources for a binary, so it will not add them all automatically.

VIRT-V2V

First a little history. Virt-v2v has been through at least two complete rewrites, so this is probably about the third version (but we don’t intend to rewrite it again). The previous version was written in Perl and can be found here: https://git.fedorahosted.org/git/virt-v2v.git

The current version started out as almost a line-for-line rewrite of the Perl code in OCaml + C, and it still has a fairly similar structure. Therefore if there are details of this code that you don’t understand (especially in the details of guest conversion), checking the Perl code may help.

The files to start with when reading this code are:

•

types.mli

•

v2v.ml

types.mli defines all the structures used and passed around when communicating between different bits of the program. v2v.ml controls how the program runs in stages.

After studying those files, you may want to branch out into the input modules (input_*), the output modules (output_*) or the conversion modules (convert_*). The input and output modules define -i and -o options (see the manual). The conversion modules define what guest types we can handle and the detailed steps involved in converting them.

Every other file in this directory is a support module / library of some sort. Some code is written in C, especially where we want to use an external C library such as libxml2.

VIRT-P2V

Virt-p2v is a front end on virt-v2v. ie. All it does is act as a GUI front end, and it calls out to virt-v2v to perform the actual conversion. Therefore most of the C code in the p2v/ subdirectory is Gtk (GUI) code, or supporting code for talking to the remote conversion server. There is no special support for physical machines in virt-v2v. They are converted in the same way as foreign VMs.

Running virt-p2v

You can run the p2v/virt-p2v binary directly, but it will try to convert your machine’s real /dev/sda which is unlikely to work well. However virt-p2v also has a test mode in which you can supply a test disk:

 make -C p2v run-virt-p2v-directly

This is a wrapper around the virt-p2v(1) –test-disk option. You can control the “physical machine” disk by setting "PHYSICAL_MACHINE" to point to a disk image.

A more realistic test is to run virt-p2v inside a VM on the local machine. To do that, do:

 make -C p2v run-virt-p2v-in-a-vm

This also runs qemu with the “physical machine” disk (which you can set by setting "PHYSICAL_MACHINE"), a virtual CD, and a variety of network cards for testing. You can change the qemu binary and add extra qemu options by setting "QEMU" and/or "QEMU_OPTIONS" on the make commandline.

A third way to run virt-p2v simulates fairly accurately the program being downloaded over PXE and then doing an automatic conversion of the source physical machine (the non-GUI path — see next section below):

 make -C p2v run-virt-p2v-non-gui-conversion

Understanding the virt-p2v code

See also: “HOW VIRT-P2V WORKS” in virt-p2v(1)

There are two paths through the code, GUI or non-GUI (parsing the kernel command line):

 main.c ──────┬─────▶ gui.c ──────┬─────▶ conversion.c
              │                   │
              │                   │
              └────▶ kernel.c ────┘

but both paths call back to the conversion.c function "start_conversion" to run the remote virt-v2v.

The main task of gui.c/kernel.c is to populate the virt-v2v configuration (config.c).

During conversion, we need to establish ssh connections, and that is done using two libraries:

 conversion.c ──────▶ ssh.c ──────▶ miniexpect.c

where ssh.c is responsible for managing ssh connections overall, and miniexpect.c implements “expect-like” functionality for talking interactively to the remote virt-v2v conversion server.

(Note that miniexpect is a separate library with its own upstream, so if you patch miniexpect.c, then please make sure the changes get reflected in miniexpect’s upstream too: http://git.annexia.org/?p=miniexpect.git;a=summary)

MAINTAINER TASKS

MAINTAINER MAKEFILE TARGETS

These "make" targets probably won’t work and aren’t useful unless you are a libguestfs maintainer.

make maintainer-commit

This commits everything in the working directory with the commit message "Version $(VERSION).". You must update configure.ac, clean and rebuild first.

make maintainer-tag

This tags the current HEAD commit with the tag "v$(VERSION)" and one of the messages:

 Version $(VERSION) stable
 Version $(VERSION) development

(See “LIBGUESTFS VERSION NUMBERS” in guestfs(3) for the difference between a stable and development release.)

make maintainer-check-authors

Check that all authors (found in git commit messages) are included in the generator/authors.ml file.

make maintainer-check-extra-dist

This rule must be run after "make dist" (so there is a tarball in the working directory). It compares the contents of the tarball with the contents of git to ensure that no files have been missed from Makefile.am "EXTRA_DIST" rules.

make maintainer-upload-website

This is used by the software used to automate libguestfs releases to copy the libguestfs website to another git repository before it is uploaded to the web server.

MAKING A STABLE RELEASE

When we make a stable release, there are several steps documented here. See “LIBGUESTFS VERSION NUMBERS” in guestfs(3) for general information about the stable branch policy.

•

Check "make && make check" works on at least:

Fedora (x86-64)

Debian (x86-64)

Ubuntu (x86-64)

Fedora (aarch64)

Fedora (ppc64)

Fedora (ppc64le)

•

Check "./configure --without-libvirt" works.

•

Finalize guestfs-release-notes.pod

•

Push and pull from Zanata.

Run:

 zanata push

to push the latest POT files to Zanata. Then run:

 ./zanata-pull.sh

which is a wrapper to pull the latest translated *.po files.

•

Consider updating gnulib to latest upstream version.

•

Create new stable and development directories under http://libguestfs.org/download.

•

Edit website/index.html.in.

•

Set the version (in configure.ac) to the new stable version, ie. 1.XX.0, and commit it:

 ./localconfigure
 make distclean -k
 ./localconfigure
 make && make dist
 make maintainer-commit
 make maintainer-tag

•

Create the stable branch in git:

 git branch stable-1.XX
 git push origin stable-1.XX

•

Do a full release of the stable branch.

•

Set the version to the next development version and commit that. Optionally do a full release of the development branch.

INTERNAL DOCUMENTATION

This section documents internal functions inside libguestfs and various utilities. It is intended for libguestfs developers only.

This section is autogenerated from "/**" comments in source files, which are marked up in POD format.

These functions are not publicly exported, and may change or be removed at any time.

Subdirectory lib

File lib/actions-support.c

Helper functions for the actions code in lib/actions-*.c.

File lib/appliance-cpu.c

The appliance choice of CPU model.

Function "lib/appliance-cpu.c:guestfs_int_get_cpu_model"

 const char *
 guestfs_int_get_cpu_model (int kvm)

Return the right CPU model to use as the qemu "-cpu" parameter or its equivalent in libvirt. This returns:

"host"

The literal string "host" means use "-cpu host".

some string

Some string such as "cortex-a57" means use "-cpu cortex-a57".

"NULL"

"NULL" means no "-cpu" option at all. Note returning "NULL" does not indicate an error.

This is made unnecessarily hard and fragile because of two stupid choices in QEMU:

•

The default for "qemu-system-aarch64 -M virt" is to emulate a "cortex-a15" (WTF?).

•

We don’t know for sure if KVM will work, but "-cpu host" is broken with TCG, so we almost always pass a broken "-cpu" flag if KVM is semi-broken in any way.

File lib/appliance-kcmdline.c

The appliance kernel command line.

Definition "lib/appliance-kcmdline.c:VALID_TERM"

#define VALID_TERM

Check that the $TERM environment variable is reasonable before we pass it through to the appliance.

Function "lib/appliance-kcmdline.c:guestfs_int_appliance_command_line"

char *
guestfs_int_appliance_command_line(guestfs_h *g, const char *appliance_dev,
int flags)

Construct the Linux command line passed to the appliance. This is used by the "direct" and "libvirt" backends, and is simply located in this file because it’s a convenient place for this common code.

The "appliance_dev" parameter must be the full device name of the appliance disk and must have already been adjusted to take into account virtio-blk or virtio-scsi; eg "/dev/sdb".

The "flags" parameter can contain the following flags logically or’d together (or 0):

"APPLIANCE_COMMAND_LINE_IS_TCG"

If we are launching a qemu TCG guest (ie. KVM is known to be disabled or unavailable). If you don’t know, don’t pass this flag.

Note that this function returns a newly allocated buffer which must be freed by the caller.

File lib/appliance-uefi.c

Find the UEFI firmware needed to boot the appliance.

See also lib/uefi.c (autogenerated file) containing the firmware file locations.

Function "lib/appliance-uefi.c:guestfs_int_get_uefi"

int guestfs_int_get_uefi(guestfs_h *g, char **code, char **vars, int *flags)

Return the location of firmware needed to boot the appliance. This is aarch64 only currently, since that’s the only architecture where UEFI is mandatory (and that only for RHEL).

*code is initialized with the path to the read-only UEFI code file. *vars is initialized with the path to a copy of the UEFI vars file (which is cleaned up automatically on exit).

If *code == *vars == "NULL" then no UEFI firmware is available.

*code and *vars should be freed by the caller.

If the function returns "-1" then there was a real error which should cause appliance building to fail (no UEFI firmware is not an error).

See also v2v/utils.ml:find_uefi_firmware

File lib/appliance.c

This file deals with building the libguestfs appliance.

Function "lib/appliance.c:guestfs_int_build_appliance"

int guestfs_int_build_appliance(guestfs_h *g,
char **kernel_rtn,
char **initrd_rtn,
char **appliance_rtn)

Locate or build the appliance.

This function locates or builds the appliance as necessary, handling the supermin appliance, caching of supermin-built appliances, or using either a fixed or old-style appliance.

The return value is 0 = good, "-1" = error. Returned in "appliance.kernel" will be the name of the kernel to use, "appliance.initrd" the name of the initrd, "appliance.image" the name of the ext2 root filesystem. "appliance.image" can be "NULL", meaning that we are using an old-style (non-ext2) appliance. All three strings must be freed by the caller. However the referenced files themselves must not be deleted.

The process is as follows:

1.

Look in "path" which contains a supermin appliance skeleton. If no element has this, skip straight to step 3.

2.

Call "supermin --build" to build the full appliance (if it needs to be rebuilt). If this is successful, return the full appliance.

3.

Check "path", looking for a fixed appliance. If one is found, return it.

4.

Check "path", looking for an old-style appliance. If one is found, return it.

The supermin appliance cache directory lives in $TMPDIR/.guestfs-$UID/ and consists of up to four files:

  $TMPDIR/.guestfs-$UID/lock            - the supermin lock file
  $TMPDIR/.guestfs-$UID/appliance.d/kernel - the kernel
  $TMPDIR/.guestfs-$UID/appliance.d/initrd - the supermin initrd
  $TMPDIR/.guestfs-$UID/appliance.d/root   - the appliance

Multiple instances of libguestfs with the same UID may be racing to create an appliance. However (since supermin ≥ 5) supermin provides a –lock flag and atomic update of the appliance.d subdirectory.

Function "lib/appliance.c:locate_or_build_appliance"

static int
locate_or_build_appliance(guestfs_h *g,
struct appliance_files *appliance,
const char *path)

Check "path", looking for one of appliances: supermin appliance, fixed appliance or old-style appliance. If one of the fixed appliances is found, return it. If the supermin appliance skeleton is found, build and return appliance.

Return values:

  1 = appliance is found, returns C<appliance>,
  0 = appliance not found,
 -1 = error which aborts the launch process.

Function "lib/appliance.c:search_appliance"

 static int
 search_appliance (guestfs_h *g, struct appliance_files *appliance)

Search elements of "g->path", returning the first "appliance" element which matches the predicate function "locate_or_build_appliance".

Return values:

  1 = a path element matched, returns C<appliance>,
  0 = no path element matched,
 -1 = error which aborts the launch process.

Function "lib/appliance.c:build_supermin_appliance"

static int
build_supermin_appliance(guestfs_h *g,
const char *supermin_path,
struct appliance_files *appliance)

Build supermin appliance from "supermin_path" to $TMPDIR/.guestfs-$UID.

Returns: 0 = built or "-1" = error (aborts launch).

Function "lib/appliance.c:run_supermin_build"

static int
run_supermin_build(guestfs_h *g,
const char *lockfile,
const char *appliancedir,
const char *supermin_path)

Run "supermin --build" and tell it to generate the appliance.

Function "lib/appliance.c:dir_contains_file"

static int
dir_contains_file(guestfs_h *g, const char *dir, const char *file)

Returns true iff "file" is contained in "dir".

Function "lib/appliance.c:dir_contains_files"

static int
dir_contains_files(guestfs_h *g, const char *dir, …)

Returns true iff every listed file is contained in "dir".

File lib/command.c

A wrapper for running external commands, loosely based on libvirt’s "virCommand" interface.

In outline to use this interface you must:

1.

Create a new command handle:

struct command *cmd;
cmd = guestfs_int_new_command(g);

2.

Either add arguments:

guestfs_int_cmd_add_arg(cmd, “qemu-img”);
guestfs_int_cmd_add_arg(cmd, “info”);
guestfs_int_cmd_add_arg(cmd, filename);

(NB: You don’t need to add a "NULL" argument at the end.)

3.

Or construct a command using a mix of quoted and unquoted strings. (This is useful for system(3)/"popen("r")"-style shell commands, with the added safety of allowing args to be quoted properly).

guestfs_int_cmd_add_string_unquoted(cmd, “qemu-img info “);
guestfs_int_cmd_add_string_quoted(cmd, filename);

4.

Set various flags, such as whether you want to capture errors in the regular libguestfs error log.

5.

Run the command. This is what does the fork(2) call, optionally loops over the output, and then does a waitpid(3) and returns the exit status of the command.

r = guestfs_int_cmd_run(cmd);
if (r == -1)
// error
// else test r using the WIF* functions

6.

Close the handle:

 guestfs_int_cmd_close (cmd);

(or use "CLEANUP_CMD_CLOSE").

Function "lib/command.c:guestfs_int_new_command"

 struct command *
 guestfs_int_new_command (guestfs_h *g)

Create a new command handle.

Function "lib/command.c:guestfs_int_cmd_add_arg"

void guestfs_int_cmd_add_arg(struct command *cmd, const char *arg)

Add single arg (for "execv"-style command execution).

Function "lib/command.c:guestfs_int_cmd_add_arg_format"

void guestfs_int_cmd_add_arg_format(struct command *cmd, const char *fs, …)

Add single arg (for "execv"-style command execution) using a printf(3)-style format string.

Function "lib/command.c:guestfs_int_cmd_add_string_unquoted"

void guestfs_int_cmd_add_string_unquoted(struct command *cmd, const char *str)

Add a string (for system(3)-style command execution).

This variant adds the strings without quoting them, which is dangerous if the string contains untrusted content.

Function "lib/command.c:guestfs_int_cmd_add_string_quoted"

void guestfs_int_cmd_add_string_quoted(struct command *cmd, const char *str)

Add a string (for system(3)-style command execution).

The string is enclosed in double quotes, with any special characters within the string which need escaping done. This is used to add a single argument to a system(3)-style command string.

Function "lib/command.c:guestfs_int_cmd_set_stdout_callback"

void guestfs_int_cmd_set_stdout_callback(struct command *cmd,
cmd_stdout_callback stdout_callback,
void *stdout_data, unsigned flags)

Set a callback which will capture stdout.

If flags contains "CMD_STDOUT_FLAG_LINE_BUFFER" (the default), then the callback is called line by line on the output. If there is a trailing "
" then it is automatically removed before the callback is called. The line buffer is "