Author: Jovan Stosic

How to Install Docker on Ubuntu 14.04 LTS

Jovan Stosic February 9, 2019

Introduction

Docker is a container-based software framework for automating deployment of applications. “Containers” are encapsulated, lightweight, and portable application modules.

Pre-Flight Check

  • These instructions are intended for installing Docker.
  • I’ll be working from a Liquid Web Core Managed Ubuntu 14.04 LTS server, and I’ll be logged in as root.

Step 1: Installation of Docker

First, you’ll follow a simple best practice: ensuring the list of available packages is up to date before installing anything new.

apt-get update

Let’s install Docker by installing the docker-io package:

apt-get -y install docker.io

Link and fix paths with the following two commands:

ln -sf /usr/bin/docker.io /usr/local/bin/docker
sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/docker.io

Finally, and optionally, let’s configure Docker to start when the server boots:

update-rc.d docker.io defaults

Step 2: Download a Docker Container

Let’s begin using Docker! Download the fedora Docker image:

docker pull ubuntu

Step 3: Run a Docker Container

Now, to setup a basic ubuntu container with a bash shell, we just run one command. docker run will run a command in a new container, -i attaches stdin and stdout, -t allocates a tty, and we’re using the standard ubuntu container.

docker run -i -t ubuntu /bin/bash

That’s it! You’re now using a bash shell inside of a ubuntu docker container.

To disconnect, or detach, from the shell without exiting use the escape sequence Ctrl-p + Ctrl-q.

There are many community containers already available, which can be found through a search. In the command below I am searching for the keyword debian:

docker search debian

Source: How to Install Docker on Ubuntu 14.04 LTS | Liquid Web Knowledge Base

How to Install Docker on Ubuntu 14.04 LTS was last modified: February 9th, 2019 by Jovan Stosic

command line – What is sed and what is it used for?

Jovan Stosic February 9, 2019

In basic usage it is used for ‘search and replace’ with strings.

echo "The quick brown fox jumps over the lazy dog" | sed 's/dog/cat/'

returns

"The quick brown fox jumps over the lazy cat"

Sed really shines when regular expressions are used with it.

Source: command line – What is sed and what is it used for? – Ask Ubuntu

command line – What is sed and what is it used for? was last modified: February 9th, 2019 by Jovan Stosic

apt-get(8) – Linux man page

Jovan Stosic February 9, 2019

apt-get(8) – Linux man page

Name

apt-get – APT package handling utility – command-line interface

Synopsis

apt-get [options] [-o config=string] [-c=cfgfile] command [pkg]

Description

apt-get is the command-line tool for handling packages, and may be considered the user’s “back-end” to other tools using the APT library. Several “front-end” interfaces exist, such as synaptic and aptitude.

Commands

Unless the -h, or –help option is given, one of the commands below must be present.

update
Used to re-synchronize the package index files from their sources. The indexes of available packages are fetched from the location(s) specified in /etc/apt/sources.list(5). An update should always be performed before an upgrade or dist-upgrade.
upgrade
Used to install the newest versions of all packages currently installed on the system from the sources enumerated in /etc/apt/sources.list(5). Packages currently installed with new versions available are retrieved and upgraded; under no circumstances are currently installed packages removed, nor are packages that are not already installed retrieved and installed. New versions of currently installed packages that cannot be upgraded without changing the install status of another package will be left at their current version. An update must be performed first so that apt-get knows that new versions of packages are available.
dist-upgrade
In addition to performing the function of upgrade, this option also intelligently handles changing dependencies with new versions of packages; apt-get has a “smart” conflict resolution system, and it will attempt to upgrade the most important packages at the expense of less important ones, if necessary.
The /etc/apt/sources.list(5) file contains a list of locations from
which to retrieve desired package files. See also apt_preferences(5) for a mechanism for over-riding the general settings for individual packages.
install pkg(s)
This option is followed by one or more packages desired for installation. Each package is a package name, not a fully qualified filename (for instance, in a Fedora Core system, glibc would be the argument provided, not glibc-2.4.8.i686.rpm).
All packages required by the package(s) specified for installation will also
be retrieved and installed. The /etc/apt/sources.list(5) file is used to locate the repositories for the desired packages. If a hyphen (–) is appended to the package name (with no intervening space), the identified package will be removed if it is currently installed. Similarly a plus sign (+) can be used to designate a package to install. These latter features may be used to override decisions made by apt-get‘s conflict resolution system.
A specific version of a package can be selected for installation by
following the package name with an equals (=) and the version of the package to select. This will cause that version to be located and selected for install. Alternatively, a specific distribution can be selected by following the package name with a slash (/) and the version of the distribution or the Archive name (i.e. stable, testing, unstable).
Both of the version selection mechanisms can downgrade packages and must be
used with care.
Finally, the apt_preferences(5) mechanism allows you to create an
alternative installation policy for individual packages.
If no package matches the given expression and the expression contains one
of “.”, “?” or “*” then it is assumed to be a POSIX regular expression, and it is applied to all package names in the database. Any matches are then installed (or removed). Note that matching is done by substring so “lo.*” matches “how-lo” and “lowest”. If this is undesired, anchor the regular expression with a “^” or “$” character, or create a more specific regular expression.
remove pkg(s)
Identical to install except that packages are removed instead of installed. If a plus sign (+) is appended to the package name (with no intervening space), the identified package will be installed instead of removed.
source source_pkg
Causes apt-get to fetch source packages. APT will examine the available packages to decide which source package to fetch. It will then find and download into the current directory the newest available version of that source package. Source packages are tracked separately from binary packages via rpm-src type lines in the sources.list(5) file. This probably will mean that you will not get the same source as the package you have installed, or could install. If the –compile options is specified then the package will be compiled to a binary using rpmbuild, if –download-only is specified then the source package will not be unpacked.
A specific source version can be retrieved by following the source name with
an equals (=) and then the version to fetch, similar to the mechanism used for the package files. This enables exact matching of the source package name and version, implicitly enabling the APT::Get::Only-Source option.
Note that source packages are not tracked like binary packages, they exist
only in the current directory and are similar to downloading source tar balls.
build-dep source_pkg
Causes apt-get to install/remove packages in an attempt to satisfy the build dependencies for a source package.
check
Diagnostic tool; it updates the package cache and checks for broken dependencies.
clean
Clears out the local repository of retrieved package files. It removes everything but the lock file from /var/cache/apt/archives/ and /var/cache/apt/archives/partial/.
autoclean
Like clean, autoclean clears out the local repository of retrieved package files. The difference is that it only removes package files that can no longer be downloaded, and are largely useless. This allows a cache to be maintained over a long period of time without it growing out of control. The configuration option APT::Clean-Installed will prevent installed packages from being erased if it is set to off.

Options

All command-line options may be set using the configuration file, the descriptions indicate the configuration option to set. For boolean options you can override the config file by using something like -f-, –no-f, -f=no or several other variations.

-d, –download-only
Download only; package files are only retrieved, not unpacked or installed.
Configuration Item: APT::Get::Download-Only.
-f, –fix-broken
Fix. Attempt to correct a system with broken dependencies in place. This option, when used with install/remove, can omit any packages to permit APT to deduce a likely solution. Any package(s) that are specified must completely correct the problem. This option is sometimes necessary when running APT for the first time; APT itself does not allow broken package dependencies to exist on a system. It is possible that a system’s dependency structure can be so corrupt as to require manual intervention. Use of this option together with -m may produce an error in some situations.
Configuration Item: APT::Get::Fix-Broken.
-m, –ignore-missing, –fix-missing
Ignore missing packages. If packages cannot be retrieved or fail the integrity check after retrieval (corrupted package files), hold back those packages and handle the result. Use of this option together with -f may produce an error in some situations. If a package is selected for installation (particularly if it is mentioned on the command-line) and it could not be downloaded then it will be silently held back.
Configuration Item: APT::Get::Fix-Missing.
–no-download
Disables downloading of packages. This is best used with –ignore-missing to force APT to use only the rpms it has already downloaded.
Configuration Item: APT::Get::Download.
-q, –quiet
Quiet. Produces output suitable for logging, omitting progress indicators. More q‘s will produce more quiet up to a maximum of two. You can also use -q=# to set the quiet level, overriding the configuration file. Note that quiet level 2 implies -y, you should never use -qq without a no-action modifier such as -d, –print-uris or -s as APT may decided to do something you did not expect.
Configuration Item: quiet.
-s, –simulate, –just-print, –dry-run, –recon, –no-act
No action. Perform a simulation of events that would occur but do not actually change the system.
Configuration Item: APT::Get::Simulate.
Simulate prints out a series of lines, each one representing an rpm
operation: Configure (Conf), Remove (Remv), Unpack (Inst). Square brackets indicate broken packages with an empty set of square brackets meaning breaks that are of no consequence (rare).
-y, –yes, –assume-yes
Automatic yes to prompts. Assume “yes” as answer to all prompts and run non-interactively. If an undesirable situation, such as changing a held package or removing an essential package, occurs then apt-get will abort.
Configuration Item: APT::Get::Assume-Yes.
-u, –show-upgraded
Show upgraded packages. Print out a list of all packages that are to be upgraded.
Configuration Item: APT::Get::Show-Upgraded.
-V, –verbose-versions
Show full versions for upgraded and installed packages.
Configuration Item: APT::Get::Show-Versions.
-b, –compile, –build
Compile source packages after downloading them.
Configuration Item: APT::Get::Compile.
–ignore-hold
Ignore package Holds. This causes apt-get to ignore a hold placed on a package. This may be useful in conjunction with dist-upgrade to override a large number of undesired holds.
Configuration Item: APT::Ignore-Hold.
–no-upgrade
Do not upgrade packages. When used in conjunction with install, no-upgrade will prevent packages listed from being upgraded if they are already installed.
Configuration Item: APT::Get::Upgrade.
–force-yes
Force yes. This is a dangerous option that will cause apt-get to continue without prompting if it is doing something potentially harmful. It should not be used except in very special situations. Using –force-yes can potentially destroy your system!
Configuration Item: APT::Get::force-yes.
–print-uris
Instead of fetching the files to install, their URIs are printed. Each URI will have the path, the destination file name, the size and the expected md5 hash. Note that the file name to write to will not always match the file name on the remote site! This also works with the source and update commands. When used with the update command, the MD5 and size are not included, and it is up to the user to decompress any compressed files.
Configuration Item: APT::Get::Print-URIs.
–reinstall
Re-Install packages that are already installed and at the newest version.
Configuration Item: APT::Get::ReInstall.
–list-cleanup
This option defaults to on, use –no-list-cleanup to turn it off. When on, apt-get will automatically manage the contents of /var/lib/apt/lists to ensure that obsolete files are erased. The only reason to turn it off is if you frequently change your source list.
Configuration Item: APT::Get::List-Cleanup.
-t, –target-release, –default-release
This option controls the default input to the policy engine. It creates a default pin at priority 990 using the specified release string. The preferences file may further override this setting. In short, this option lets you have simple control over which distribution packages will be retrieved from. Some common examples might be -t ‘2.1*’ or -t unstable.
Configuration Item: APT::Default-Release; see also the
apt_preferences(5) manual page.
–trivial-only
Only perform operations that are “trivial”. Logically this can be considered related to –assume-yes. Where –assume-yes will answer yes to any prompt, –trivial-only will answer no.
Configuration Item: fIAPT::Get::Trivial-Only.
–no-remove
If any packages are to be removed apt-get immediately aborts without prompting.
Configuration Item: APT::Get::Remove.
–only-source
Only has meaning for the source command. Indicates that the given source names are not to be mapped through the binary table. This means that if this option is specified, the source command will only accept source package names as arguments, rather than accepting binary package names and looking up the corresponding source package.
Configuration Item: APT::Get::Only-Source.
-h, –help
Show a short usage summary.
-v, –version
Show the program version.
-c, –config-file
Configuration File. Specify a configuration file to use. The program will read the default configuration file and then this configuration file. See apt.conf(5) for syntax information.
-o, –option
Set a Configuration Option. This will set an arbitrary configuration option. The syntax is -o Foo::Bar=bar.

Source: apt-get(8) – Linux man page

apt-get(8) – Linux man page was last modified: February 9th, 2019 by Jovan Stosic

Move existing WordPress site into Docker – Dots and Brackets: Code Blog

Jovan Stosic February 9, 2019

Imagine you have the following existing python2 application “hello.py” with the following content:

print "hello"

You have to do the following things to dockerize this application:

Create a folder where you’d like to store your Dockerfile in.

Create a file named “Dockerfile”

The Dockerfile consists of several parts which you have to define as described below:

Like a VM, an image has an operating system. In this example, I use ubuntu 16.04. Thus, the first part of the Dockerfile is:

FROM ubuntu:16.04

Imagine you have a fresh Ubuntu – VM, now you have to install some things to get your application working, right? This is done by the next part of the Dockerfile:

RUN     apt-get update && \ 
        apt-get upgrade -y && \
        apt-get install -y python

For Docker, you have to create a working directory now in the image. The commands that you want to execute later on to start your application will search for files (like in our case the python file) in this directory. Thus, the next part of the Dockerfile creates a directory and defines this as the working directory:

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

As a next step, you copy the content of the folder where the Dockerfile is stored in to the image. In our example, the hello.py file is copied to the directory we created in the step above.

COPY . /usr/src/app

Finally, the following line executes the command “python hello.py” in your image:

CMD [ "python", "hello.py" ]

The complete Dockerfile looks like this:

FROM ubuntu:16.04

RUN     apt-get update && \
        apt-get upgrade -y && \
        apt-get install -y python 

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

COPY . /usr/src/app

CMD [ "python", "hello.py" ]

Save the file and build the image by typing in the terminal:

$ docker build -t hello .

This will take some time. Afterwards, check if the image “hello” how we called it in the last line has been built successfully:

$ docker images

Run the image:

docker run hello

The output shout be “hello” in the terminal.

This is a first start. When you use Docker for web applications, you have to configure ports etc.

https://codeblog.dotsandbrackets.com/migrate-wordpress-docker/

Move existing WordPress site into Docker – Dots and Brackets: Code Blog was last modified: February 10th, 2019 by Jovan Stosic

virtual machine – Dynamic memory allocation in KVM – Stack Overflow

Jovan Stosic February 9, 2019

I have Ubuntu 16.04 host over which I installed a Virtual guest (Windows server 2012) using KVM. The total RAM available is 16 GB.

I have installed virtio balloon drivers. I have allocated 4GB current memory and 12GB maximum memory to the windows. However the windows sees 8GB unallocated memory as used memory. Please see the attached figure. Memory usage in Guest

When I open some heavy applications the memory assigned is limited to the available memory from 4GB. The computer behaves in the same manner when full memory is being utilized in the windows (does not run smoothly).

The windows can see all the 12 GB RAM but can only utilize 4 GB. How do I ensure that windows can utilize all off the 12GB of memory?

———————–
The behaviour you describe is simply the way the balloon driver operates. The hypervisor exposes a virtual machine with 12 GB of virtual DIMMs installed. The balloon driver then grabs 8 GB of this memory and gives it back to the hypervisor. The way it grabs memory varies per guest OS, but essentially it has to allocate it to make it appear to be in use, thus preventing the guest OS using it. The balloon driver does not automatically adjust depending on guest workload requirements. So if your windows guests needs more than 4 GB, you need to use libvirt/virsh on the host OS to change the balloon level, to give some of the “used” 8 GB back to the guest OS. If you want the guest OS to see & use the full 12 GB, simply don’t use the balloon driver at all.

https://stackoverflow.com/questions/43039272/dynamic-memory-allocation-in-kvm

virtual machine – Dynamic memory allocation in KVM – Stack Overflow was last modified: February 9th, 2019 by Jovan Stosic

Virtual Box – Memory Ballooning – Guest Additions

Jovan Stosic February 9, 2019

The Guest Additions can change the amount of host memory that a VM uses, while the machine is running. Because of how this is implemented, this feature is called memory ballooning.

Note

  • Oracle VM VirtualBox supports memory ballooning only on 64-bit hosts. It is not supported on Mac OS X hosts.

  • Memory ballooning does not work with large pages enabled. To turn off large pages support for a VM, runVBoxManage modifyvm <VM name> --largepages off

Normally, to change the amount of memory allocated to a virtual machine, you have to shut down the virtual machine entirely and modify its settings. With memory ballooning, memory that was allocated for a virtual machine can be given to another virtual machine without having to shut the machine down.

When memory ballooning is requested, the Oracle VM VirtualBox Guest Additions, which run inside the guest, allocate physical memory from the guest operating system on the kernel level and lock this memory down in the guest. This ensures that the guest will not use that memory any longer. No guest applications can allocate it, and the guest kernel will not use it either. Oracle VM VirtualBox can then reuse this memory and give it to another virtual machine.

The memory made available through the ballooning mechanism is only available for reuse by Oracle VM VirtualBox. It is not returned as free memory to the host. Requesting balloon memory from a running guest will therefore not increase the amount of free, unallocated memory on the host. Effectively, memory ballooning is therefore a memory overcommitment mechanism for multiple virtual machines while they are running. This can be useful to temporarily start another machine, or in more complicated environments, for sophisticated memory management of many virtual machines that may be running in parallel depending on how memory is used by the guests.

At this time, memory ballooning is only supported throughVBoxManage. Use the following command to increase or decrease the size of the memory balloon within a running virtual machine that has Guest Additions installed:

VBoxManage controlvm "VM name" guestmemoryballoon n

where VM name is the name or UUID of the virtual machine in question and n is the amount of memory to allocate from the guest in megabytes. See Section 8.14, “VBoxManage controlvm”.

You can also set a default balloon that will automatically be requested from the VM every time after it has started up with the following command:

VBoxManage modifyvm "VM name" --guestmemoryballoon n

By default, no balloon memory is allocated. This is a VM setting, like other modifyvm settings, and therefore can only be set while the machine is shut down. See Section 8.8, “VBoxManage modifyvm”.

http://www.virtualbox.org/manual/ch04.html#guestadd-balloon

Virtual Box – Memory Ballooning – Guest Additions was last modified: February 9th, 2019 by Jovan Stosic