Yearly Archives: 2018

Docker container utilization monitoring

An article explaining Docker container utilization monitoring. How to monitor or save reports of Docker container resource utilization and how to format output according to your requirement.

Docker containers are processes running on host OS using its resources. It means Docker containers are using CPU, Memory, and IO from Host OS to execute their commands or perform their tasks. Resource utilization is a major area leading to the performance of a server or application.

Host OS being a Linux in our case can be monitored using tools like sar, top, etc for resource utilization. You can trace down PID of Docker containers and then drill down to those PID’s utilization in the host’s monitoring tool to get container utilization. But this is a bit tedious job and not feasible in case you have the number of containers running on your server. Docker already took care of it and provided its own real-time monitoring tool which reports resource utilization by each container in real-time.

If you still don’t have Docker on your system, read here how to install Docker in Linux and 8 basic Docker management commands.

How to monitor Docker container utilization?

Docker provided command stats to provide real-time container’s resource utilization statistics. Commands run in terminal like top command and update values in real-time.

Read all docker or containerization related articles here from KernelTalk’s archives.

You can supply a container ID or name to this command to view the statistics of that specific container. If no container name/ID supplied, it will show stats of all running containers.

root@kerneltalks # docker container stats
CONTAINER ID        NAME                    CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
2554070a4ba7        friendly_hodgkin        0.19%               205MiB / 991MiB     20.69%              1.21kB / 767B       105MB / 8.7kB       31
b60fa988daee        condescending_galileo   0.18%               201MiB / 991MiB     20.29%              1.21kB / 761B       96.3MB / 9.22kB     31

root@kerneltalks # docker container stats friendly_hodgkin
CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
2554070a4ba7        friendly_hodgkin    0.15%               205.2MiB / 991MiB   20.71%              1.21kB / 767B       105MB / 8.7kB       31

The output is tabulated and column-wise it has –

Container ID: Docker container ID
Name: Docker container name
CPU %: CPU percentage of the host being utilized by container right now.
MEM USAGE / LIMIT: Memory being utilized by container right now / Max memory which can be used by the container
MEM %: Memory percentage of Host being utilized by container right now.
NET I/ O: Network Input Output traffic on container’s network interface
BLOCK I/ O: Disk IO did on the Host storage
PIDS: Total number of processes/threads container created/forked.

You have to press cntrl+c to return to a prompt from real-time updating stats screen.

How to save Docker container utilization?

Now, if you want to save container utilization or you want to use stats command in some script then you may want to run it for 1 iteration only and exits automatically rather than keep running.

In such a case, you need to use --no-stream switch along with stats command.

root@kerneltalks # docker container stats --no-stream
CONTAINER ID        NAME                    CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
2554070a4ba7        friendly_hodgkin        0.15%               205.2MiB / 991MiB   20.71%              1.21kB / 767B       105MB / 8.7kB       31
b60fa988daee        condescending_galileo   0.15%               201.3MiB / 991MiB   20.31%              1.21kB / 761B       96.3MB / 9.22kB     31
root@kerneltalks #

You can redirect this output to file for further processing.

CPU and Memory utilization of Docker container

stats command offers to format according to your need by using --format switch. It has GO template formatting available with this switch.

Using it, you can make stats command to display only CPU and MEM utilization of containers like below :

root@kerneltalks # docker container stats --format "table {{.Container}}\t{{.CPUPerc}}\t{{.MemPerc}}"
CONTAINER           CPU %               MEM %
2554070a4ba7        0.18%               20.71%
b60fa988daee        0.18%               20.32%

Placeholders of this formatting are as below –

.Container Container name or ID (user input)
.Name Container name
.ID Container ID
.CPUPerc CPU %
.MemPerc Memory %
.MemUsage Memory usage
.NetIO Network IO
.BlockIO Block IO
.PIDs Number of PIDs

So you can format the output the way you want it and with only values, you are interested in. Then you can use no-stream and get the utilization figures to another file or pipe them to other commands for further processing.

How to import VG using different VG name

Leave a reply

Learn how to import VG with a different name. Useful when the system has the same name VG already existing and you are trying to import VG there.

In this tutorial, we will walk you through the process to import VG with a different name. During migrations, data movement activities, etc you may face issues in the VG import process. Sometimes you have disks with VGs exported on them & the same VG name is exiting on the server where you try to import VG from those disks.

In such cases, where you try to import VG when same VG name already existing in the system you get below error :

# vgimport vg02 /dev/xvdg
  Multiple VGs found with the same name: skipping vg01
  Use --select vg_uuid=<uuid> in place of the VG name.

Now, you have two choices to import VG in such case.

Import using VG UUID while keeping VG name the same. In this case, you will have 2 VG on the system with the same name and administer them & mounting LVs in them is havoc
Use vgimportclone command and import VG using a different name.

Import VG using the same name with UUID

If you took the VG map file while exporting VG then you can get VG UUID from it or you must have noted it down from vgdisplay output as well. This process also answers for how to import VG with the duplicate names!

# vgimport  --select vg_uuid=kzwQmc-qIzF-IZcv-9eW8-ToA1-mkDV-fLari2
  Volume group "vg02" successfully imported

Now you see vg02 is imported. But we already had VG named vg02 in our system. So, now we have two VGs with the same name in our system.

# vgdisplay
  --- Volume group ---
  VG Name               vg02
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               1020.00 MiB
  PE Size               4.00 MiB
  Total PE              255
  Alloc PE / Size       0 / 0
  Free  PE / Size       255 / 1020.00 MiB
  VG UUID               aRmosT-KZCY-5k40-DGSK-3WGd-ZwWD-eEXXgk

  --- Volume group ---
  VG Name               vg02
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  5
  VG Access             read/write
  VG Status             exported/resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <5.00 GiB
  PE Size               4.00 MiB
  Total PE              1279
  Alloc PE / Size       25 / 100.00 MiB
  Free  PE / Size       1254 / <4.90 GiB
  VG UUID               kzwQmc-qIzF-IZcv-9eW8-ToA1-mkDV-fLari2

Now the real challenge is to mount LV within them. You can do that but to ease out your pain, you can simply rename newly imported VG to some different name. You should use VG UUID again in vgrename command.

# vgrename kzwQmc-qIzF-IZcv-9eW8-ToA1-mkDV-fLari2 vg03
  Processing VG vg02 because of matching UUID kzwQmc-qIzF-IZcv-9eW8-ToA1-mkDV-fLari2
  Volume group "kzwQmc-qIzF-IZcv-9eW8-ToA1-mkDV-fLari2" successfully renamed to "vg03"

Once you renamed newly imported VG, your existing and new VG has different names as well. Now, you will be able to mount LVs within them without much hassle.

Hence its recommended in such a scenario you must import VG with a different name.

Import VG with the different name using vgimportclone command

vgimportclone command should be supplied with a new name with which you want VG to be imported with switch -n followed by disk name on which you exported VG

# vgimportclone -n vg01 /dev/xvdg
  VG vg02 is exported, use the --import option.

So, the command here detected that VG was exported and asks us to confirm import operation with --import switch. Let’s do that –

# vgimportclone --import -n vg01 /dev/xvdg

And command ran successfully. It won’t print any message on the terminal. You can verify that new VG on the system using vgdisplay command.

If you are looking for answers for how to change VG name during import? or how to import VG with a new name? then vgimportclone is the answer!

That’s it. You successfully imported VG with a different name. Since existing VG and your imported VG has different names now you won’t face any hurdles in mounting LVs from both of them.

How to execute command inside Docker container

Leave a reply

Learn how to access shell and execute a command inside a Docker container. Explains running commands inside already running containers or while launching containers.

If you are following the Docker series on my blog then you must have been gone through Docker basics and Docker container maintenance commands by now. In this tutorial, we will walk you through how to access shell inside Docker container and how to execute commands inside the container.

First of all, you can not execute commands or access shells in any container. Basically, the container image you are using to launch your container should have a shell in it. If the image does not support shell then you can not do anything inside the container during launch or even after launch.

Read all docker or containerization related articles here from KernelTalk’s archives.

For example, if you are launching a container from Nginx image i.e. web-server container then you won’t be able to access the shell or execute the command within it. Since its just a web-server process! But, if you are launching a container from the ubuntu image or alpine image then you will be able to access its shell since those images/software does support shell.

You can access shell inside a docker container and execute commands inside container either of using two ways –

Execute bash shell while launching container
Use docker command to execute single command inside container

Remember, each Docker image has a default command defined in it which it executes whenever it launches any container. You can edit it anytime but if you want to change it on the fly then you need to specify it at the end of the run command. So, image ignores default defined command and it executes a command specified in docker run command after it launches container.

Access shell & execute command in Docker container while launching it

Once you are confirmed that the image you are using to launch container does support shell (mostly its bash) then you need to launch a container using -it switch. where –

-i is the interactive mode.It keeps STDIN open even if you choose to detach container after launch
-t is to assign pseudo-terminal through which STDIN is kept open for user input.

I launched Ubuntu container with -it switch and I presented with shell prompt within. Observe output below –

root@kerneltalks# docker container run -it ubuntu:latest
root@2493081de86f:/# hostname
2493081de86f
root@2493081de86f:/# ls -lrt
total 20
drwxr-xr-x.   2 root root    6 Apr 24 08:34 home
drwxr-xr-x.   2 root root    6 Apr 24 08:34 boot
drwxr-xr-x.   8 root root   96 Apr 26 21:16 lib
drwxr-xr-x.  10 root root 4096 Apr 26 21:16 usr
drwxr-xr-x.   2 root root    6 Apr 26 21:16 srv
drwxr-xr-x.   2 root root    6 Apr 26 21:16 opt
drwxr-xr-x.   2 root root    6 Apr 26 21:16 mnt
drwxr-xr-x.   2 root root    6 Apr 26 21:16 media
drwxr-xr-x.   2 root root   34 Apr 26 21:16 lib64
drwx------.   2 root root   37 Apr 26 21:17 root
drwxr-xr-x.  11 root root 4096 Apr 26 21:17 var
drwxr-xr-x.   2 root root 4096 Apr 26 21:17 bin
drwxrwxrwt.   2 root root    6 Apr 26 21:17 tmp
drwxr-xr-x.   2 root root 4096 Apr 27 23:28 sbin
drwxr-xr-x.   5 root root   58 Apr 27 23:28 run
dr-xr-xr-x.  13 root root    0 Jun  2 14:40 sys
drwxr-xr-x.  29 root root 4096 Jun  2 14:58 etc
dr-xr-xr-x. 114 root root    0 Jun  2 14:58 proc
drwxr-xr-x.   5 root root  360 Jun  2 14:58 dev
root@2493081de86f:/# date
Sat Jun  2 15:00:17 UTC 2018
root@2493081de86f:/# exit

With the output, you can see after the container is launched promptly is given root@2493081de86f . Now you are within the container with root the account. Keep in mind everything inside the container happens with root id. If you see hostname of Ubuntu container is set the same as container ID. I executed a couple of commands inside the container in the above output.

Keep in mind, since the container is aimed to be very lightweight they always consist of minimal software inside. So if you are running any Linux distribution container, you won’t be able to run all commands as you would normally do in VM or Linux server.

Execute command inside already running container

The above process is applicable for the container you are about to launch. But what if you want to execute a command on the container which is already running on the system. Docker provided exec switch to access running container shell. Syntax is docker container exec <container name/ID> <command to run>

I have already a ubuntu container running in my system. I used exec switch to execute hostname, date and df commands inside the container.

root@kerneltalks # docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ae0721fb8ecf ubuntu:latest "/bin/bash" 2 minutes ago Up 2 minutes loving_bohr

root@kerneltalks # docker container exec ae0721fb8ecf date
Sat Jun 2 15:41:24 UTC 2018
root@kerneltalks # docker container exec ae0721fb8ecf hostname
ae0721fb8ecf
root@kerneltalks # docker container exec ae0721fb8ecf df
Filesystem                                                                                         1K-blocks    Used Available Use% Mounted on
/dev/mapper/docker-202:1-26198093-57ab60113158ca3f51c470fefb25a3fdf154a5309f05f254c660dba2a55dbab7  10474496  109072  10365424   2% /
tmpfs                                                                                                  65536       0     65536   0% /dev
tmpfs                                                                                                 507368       0    507368   0% /sys/fs/cgroup
/dev/xvda1                                                                                           8376320 5326996   3049324  64% /etc/hosts
shm                                                                                                    65536       0     65536   0% /dev/shm
tmpfs                                                                                                 507368       0    507368   0% /proc/scsi
tmpfs                                                                                                 507368       0    507368   0% /sys/firmware

Observe about output and all 3 commands ran successfully inside container and shown output on our host machine terminal.

8 basic Docker container management commands

3 Replies

Learn basic Docker container management with the help of these 8 commands. A useful guide for Docker beginners which includes sample command outputs.

In this article we will walk you through 6 basic Docker container commands which are useful in performing basic activities on Docker containers like run, list, stop, view logs, delete, etc. If you are new to the Docker concept then do check our introduction guide to know what is Docker & how-to guide to install Docker in Linux. Without further delay lets directly jump into commands.

How to run Docker container?

As you know, the Docker container is just an application process running on the host OS. For Docker container, you need a image to run from. Docker image when runs as process called a Docker container. You can have Docker image available locally or you have to download it from Docker hub. Docker hub is a centralized repository that has public and private images stored to pull from. Docker’s official hub is at hub.docker.com. So whenever you instruct the Docker engine to run a container, it looks for image locally, and if not found it pulls it from Docker hub.

Read all docker or containerization related articles here from KernelTalk’s archives.

Let’s run a Docker container for Apache web-server i.e httpd process. You need to run the command docker container run. The old command was just docker run but lately, Docker added sub-command section so new versions support below command –

root@kerneltalks # docker container run -d -p 80:80 httpd
Unable to find image 'httpd:latest' locally
latest: Pulling from library/httpd
3d77ce4481b1: Pull complete
73674f4d9403: Pull complete
d266646f40bd: Pull complete
ce7b0dda0c9f: Pull complete
01729050d692: Pull complete
014246127c67: Pull complete
7cd2e04cf570: Pull complete
Digest: sha256:f4610c3a1a7da35072870625733fd0384515f7e912c6223d4a48c6eb749a8617
Status: Downloaded newer image for httpd:latest
c46f2e9e4690f5c28ee7ad508559ceee0160ac3e2b1688a61561ce9f7d99d682

Docker run command takes image name as a mandatory argument along with many other optional ones. Commonly used arguments are –

-d : Detach container from the current shell
-p X:Y : Bind container port Y with host’s port X
--name : Name your container. If not used, it will be assigned randomly generated name
-e : Pass environmental variables and their values while starting a container

In the above output you can see, we supply httpd as an image name to run a container from. Since the image was not locally found, the Docker engine pulled it from Docker Hub. Now, observe it downloaded image httpd:latest where: is followed by version. That’s the naming convention of Docker container image. If you want a specific version container to run from then you can provide a version name along with image name. If not supplied, the Docker engine will always pull the latest one.

The very last line of output shown a unique container ID of your newly running httpd container.

How to list all running Docker containers?

Now, your container is running, you may want to check it or you want to list all running containers on your machine. You can list all running containers using docker container ls command. In the old Docker version, docker ps does this task for you.

root@kerneltalks # docker container ls
CONTAINER ID        IMAGE               COMMAND              CREATED             STATUS              PORTS                NAMES
c46f2e9e4690        httpd               "httpd-foreground"   11 minutes ago      Up 11 minutes       0.0.0.0:80->80/tcp   cranky_cori

Listing output is presented in column-wise format. Where column-wise values are –

Container ID: First few digits of the unique container ID
Image: Name of the image used to run the container
Command: Command ran by container after it ran
Created: Time created
Status: Current status of the container
Ports: Port binding details with host’s ports
Names: Name of the container (since we haven’t named our container you can see randomly generated name assigned to our container)

How to view logs of Docker container?

Since during the first step we used -d switch to detach container from the current shell once it ran its running in the background. In this case, we are clueless about what’s happening inside the container. So to view logs of the container, Docker provided logs command. It takes a container name or ID as an argument.

root@kerneltalks # docker container logs cranky_cori
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
[Thu May 31 18:35:07.301158 2018] [mpm_event:notice] [pid 1:tid 139734285989760] AH00489: Apache/2.4.33 (Unix) configured -- resuming normal operations
[Thu May 31 18:35:07.305153 2018] [core:notice] [pid 1:tid 139734285989760] AH00094: Command line: 'httpd -D FOREGROUND'

I used the container name in my command as an argument. You can see the Apache related log within our httpd container.

How to identify Docker container process?

The container is a process that uses host resources to run. If it’s true, then you will be able to locate the container process on the host’s process table. Let’s see how to check the container process on the host.

Docker used famous top command as its sub-commands name to view processes spawned by the container. It takes the container name/ID as an argument. In the old Docker version, only docker top command works. In newer versions, docker top and docker container top both works.

root@kerneltalks # docker container top  cranky_cori
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
root                15702               15690               0                   18:35               ?                   00:00:00            httpd -DFOREGROUND
bin                 15729               15702               0                   18:35               ?                   00:00:00            httpd -DFOREGROUND
bin                 15730               15702               0                   18:35               ?                   00:00:00            httpd -DFOREGROUND
bin                 15731               15702               0                   18:35               ?                   00:00:00            httpd -DFOREGROUND

root@kerneltalks # ps -ef |grep -i 15702
root     15702 15690  0 18:35 ?        00:00:00 httpd -DFOREGROUND
bin      15729 15702  0 18:35 ?        00:00:00 httpd -DFOREGROUND
bin      15730 15702  0 18:35 ?        00:00:00 httpd -DFOREGROUND
bin      15731 15702  0 18:35 ?        00:00:00 httpd -DFOREGROUND
root     15993 15957  0 18:59 pts/0    00:00:00 grep --color=auto -i 15702

In the first output, the list of processes spawned by that container. It has all details like use, PID, PPID, start time, command, etc. All those PID you can search in your host’s process table and you can find them there. That’s what we did in the second command. So, this proves containers are indeed just processes on Host’s OS.

How to stop Docker container?

It’s simple stop command! Again it takes container name /ID as an argument.

root@kerneltalks # docker container stop cranky_cori
cranky_cori

How to list stopped or not running Docker containers?

Now we stopped our container if we try to list container using ls command, we won’t be able to see it.

root@kerneltalks # docker container ls
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

So, in this case, to view stopped or nonrunning container you need to use -a switch along with ls command.

root@kerneltalks # docker container ls -a
CONTAINER ID        IMAGE               COMMAND              CREATED             STATUS                     PORTS               NAMES
c46f2e9e4690        httpd               "httpd-foreground"   33 minutes ago      Exited (0) 2 minutes ago                       cranky_cori

With -a switch we can see stopped container now. The notice status of this container is mentioned ‘Exited’. Since the container is just a process its termed as ‘exited’ rather than stopped!

How to start Docker container?

Now, we will start this stopped container. There is a difference between running and starting a container. When you run a container, you are starting a command in a fresh container. When you start a container, you are starting an old stopped container which has an old state saved in it. It will start it from that state forward.

root@kerneltalks #  docker container start c46f2e9e4690
c46f2e9e4690

root@kerneltalks # docker container ls -a
CONTAINER ID        IMAGE               COMMAND              CREATED             STATUS              PORTS                NAMES
c46f2e9e4690        httpd               "httpd-foreground"   35 minutes ago      Up 8 seconds        0.0.0.0:80->80/tcp   cranky_cori

How to remove Docker container?

To remove the container from your Docker engine use rm command. You can not remove the running containers. You have to first stop the container and then remove it. You can remove it forcefully using -f switch with rm command but that’s not recommended.

root@kerneltalks # docker container rm cranky_cori
cranky_cori
root@kerneltalks # docker container ls -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

You can see once we remove container, its not visible in ls -a listing too.

What is Docker? Introduction guide to Docker

Leave a reply

What is Docker? Introduction guide to Docker for beginners.

Docker! It’s a kind of hot cake right now in the IT industry. Docker is a thing now! If you are into system administration, IT operations, developments, or DevOps then at some point in time you may have or will come across work Docker and you wonder what is docker? Why is docker so famous? So, in this small introduction guide to Docker, we will explain to you about Docker.

Read all docker or containerization related articles here from KernelTalk’s archives.

What is Docker?

Docker is another layer of virtualization where virtualization happens at the operating system level. It’s a software container platform and currently leading this sector globally. You must be familiar with VMware which is virtualization at bare metal level but docker takes one step forward and virtualize things at OS level and hence removing all hardware management, capacity planning, resource management, etc. VMware runs a number of virtual machines (VMs) on single server hardware (refer Figure 1) whereas Docker runs a number of containers on a single Operating System (refer Figure 2). So in simple terms, Docker containers are just processes sharing a host operating system to perform their tasks.

Lets quickly run through the difference between VM and Docker containers. I tabulated the difference for a quick read.

Virtual machine v/s Docker container

Virtual Machine	Docker container
Its a mini version of physical machine	Its just a process
Runs on hypervisor virtualization	Runs on Linux. (HyperV needed if you run on Windows/MAC)
Has its own guest OS	No OS
Can be used only after guest OS boot finishes	Immediately ready to use when launched
Slow	Fast
Uses hardware resources of Host	Uses only OS resources like binaries/libraries of Host
Resource management needed	No resource management
It runs as long as admin/guest OS doesnt power it off	It runs as long as command runs which container executed at startup.
VM stops when you shutdown guest OS	Once the command exits, container stops

Docker engine mainly runs on Linux. So if you are running Docker on Windows or MAC then it’s actually running tiny Linux VM in the background on your Windows or MAC and on top of it, it’s running its own engine to provide you Docker functionalities on non-Linux platform.

Since Docker engine runs containers it also termed as containerization!

Why use Docker?

Docker containers are portable. They can be stored as an image which can be copied to any other machine and can be launched there. This ensures even if host OS parameters, version changes containers still functions the same across the different OS.

Containers use the host operating system, they don’t have their own OS to boot when containers are launched. It means they are almost available for use immediately as there is not booting of OS of anything that sort which takes time to prepare the container for use. Docker containers are fast to use!

They use resources from host OS, there is no resource management like adding/removing CPU, memory, storage, etc tasks on containers!

There are lots of functionality, flexibility being added to Docker every month. Its fast-evolving virtualization concept and gives you more ease of managing IT infra.

What are Docker variants available to use?

Docker Editions

At present, there are two editions available. CE and EE. CE stands for Community Edition and EE stands for Enterprise Edition. Let’s see the difference between Docker CE and Docker EE.

Docker CE	Docker EE
Community Edition	Enterprise Edition
It’s free	It’s paid
Primarily for development use	Use this edition for Production environment
Do it yourself. No support	Support subscription from Docker
For personal use	For enterprise/big/production use

Docker releases

Docker also releases in two forms. Stable and Edge. Let’s see the difference between Docker stable release and Docker edge release.

Docker stable release	Docker edge release
Its tested final release	Its kind of beta release
Stable version.	Includes upcoming features/functionalities
For dev/prod use	For experimental use only
New release every quarter	New release every month
Support available	No support for issues faced

Where to get Docker?

Docker can be downloaded from Docker’s official store. For each platform, related instructions are included. Detailed installation steps and other information on Docker can be found on Docker’s official documentation portal. You can also refer to our article to install Docker on Linux.

I believe that should be enough for an introductory article on Docker. If you have any questions/feedback, please leave us to comment below or reach us using the contact form.

How to install docker in Linux

Leave a reply

Learn how to install Docker in Linux. Docker is the next step of virtualization which does Operating system level virtualization also known as containerization.

In this article, we will walk you through the procedure to install Docker in any Linux distro like RHEL, SUSE, OEL, CentOS, Debian, Fedora, Ubuntu, etc. Sometimes your package manager like YUM or apt-get may offer package docker* to install docker on your server but it’s always good to get a fresh Docker setup. Since Docker is changing fast and it’s always advisable to install the latest version of Docker which might not be available with your package manager.

Read all docker or containerization related articles here from KernelTalk’s archives.

Install docker using package

If your package manager has a Docker package available to install then it’s an easy way to get Docker on your system.

Before going got Docker installation you should install below packages on your system to use the full flexible functionality of Docker. These packages are not dependencies but its good to have them pre-installed so that all Docker functions/drivers you can use.

For CenOs, Redhat etc YUM based systems – yum-utils device-mapper-persistent-data lvm2
For Debian, Ubuntu etc apt based systems – apt-transport-https ca-certificates curl software-properties-common

But you may not be getting the latest version of Docker in this case. You can install a package simply using yum or apt-get command. Below sample output for your reference from the OpenSuse server.

root@kerneltalks # zypper in docker
Building repository 'openSUSE-13.2-Update' cache .................................................................................................................[done]
Retrieving repository 'openSUSE-13.2-Update-Non-Oss' metadata ....................................................................................................[done]
Building repository 'openSUSE-13.2-Update-Non-Oss' cache .........................................................................................................[done]
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following NEW package is going to be installed:
  docker

1 new package to install.
Overall download size: 6.2 MiB. Already cached: 0 B  After the operation, additional 22.9 MiB will be used.
Continue? [y/n/? shows all options] (y): y
Retrieving package docker-1.9.1-56.1.x86_64                                                                                        (1/1),   6.2 MiB ( 22.9 MiB unpacked)
Retrieving: docker-1.9.1-56.1.x86_64.rpm .............................................................................................................[done (2.5 MiB/s)]
Checking for file conflicts: .....................................................................................................................................[done]
(1/1) Installing: docker-1.9.1-56.1 ..............................................................................................................................[done]
Additional rpm output:
creating group docker...
Updating /etc/sysconfig/docker...

Install docker using the script

In the below procedure, we will be using the script from Docker’s official website which will scan your system for details and automatically fetch the latest and compatible docker version for your system and installs it. We will be fetching script from this docker URL and using it to install the latest Docker on the list of Linux distros.

Fetch the latest script from docker official website using curl. If you read this script, SUPPORT_MAP variable shows the list of Linux distros this script support. If you are running any other Linux version than listed here then this method won’t be useful for you.

root@kerneltalks # curl -fsSL get.docker.com -o get-docker.sh
root@kerneltalks # ls -lrt
-rw-r--r--. 1 root root 13847 May 30 18:59 get-docker.sh

Now we have latest get-docker.sh script from docker official website on our server. Now, you just have to run the script and it will do the rest!

# sh get-docker.sh
# Executing docker install script, commit: 36b78b2
+ sh -c 'yum install -y -q yum-utils'
Package yum-utils-1.1.31-45.el7.noarch already installed and latest version
+ sh -c 'yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo'
Loaded plugins: fastestmirror
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
+ '[' edge '!=' stable ']'
+ sh -c 'yum-config-manager --enable docker-ce-edge'
Loaded plugins: fastestmirror
========================================================================= repo: docker-ce-edge =========================================================================
[docker-ce-edge]
async = True
bandwidth = 0
base_persistdir = /var/lib/yum/repos/x86_64/7
baseurl = https://download.docker.com/linux/centos/7/x86_64/edge
cache = 0
cachedir = /var/cache/yum/x86_64/7/docker-ce-edge
check_config_file_age = True
compare_providers_priority = 80
cost = 1000
deltarpm_metadata_percentage = 100
deltarpm_percentage =
enabled = 1
enablegroups = True
exclude =
failovermethod = priority
ftp_disable_epsv = False
gpgcadir = /var/lib/yum/repos/x86_64/7/docker-ce-edge/gpgcadir
gpgcakey =
gpgcheck = True
gpgdir = /var/lib/yum/repos/x86_64/7/docker-ce-edge/gpgdir
gpgkey = https://download.docker.com/linux/centos/gpg
hdrdir = /var/cache/yum/x86_64/7/docker-ce-edge/headers
http_caching = all
includepkgs =
ip_resolve =
keepalive = True
keepcache = False
mddownloadpolicy = sqlite
mdpolicy = group:small
mediaid =
metadata_expire = 21600
metadata_expire_filter = read-only:present
metalink =
minrate = 0
mirrorlist =
mirrorlist_expire = 86400
name = Docker CE Edge - x86_64
old_base_cache_dir =
password =
persistdir = /var/lib/yum/repos/x86_64/7/docker-ce-edge
pkgdir = /var/cache/yum/x86_64/7/docker-ce-edge/packages
proxy = False
proxy_dict =
proxy_password =
proxy_username =
repo_gpgcheck = False
retries = 10
skip_if_unavailable = False
ssl_check_cert_permissions = True
sslcacert =
sslclientcert =
sslclientkey =
sslverify = True
throttle = 0
timeout = 30.0
ui_id = docker-ce-edge/x86_64
ui_repoid_vars = releasever,
   basearch
username =

+ sh -c 'yum makecache'
Loaded plugins: fastestmirror
base                                                                                                                                             | 3.6 kB  00:00:00
docker-ce-edge                                                                                                                                   | 2.9 kB  00:00:00
docker-ce-stable                                                                                                                                 | 2.9 kB  00:00:00
epel/x86_64/metalink                                                                                                                             |  21 kB  00:00:00
extras                                                                                                                                           | 3.4 kB  00:00:00
updates                                                                                                                                          | 3.4 kB  00:00:00
(1/15): docker-ce-stable/x86_64/filelists_db                                                                                                     | 7.7 kB  00:00:03
(2/15): base/7/x86_64/other_db                                                                                                                   | 2.5 MB  00:00:04
(3/15): docker-ce-edge/x86_64/filelists_db                                                                                                       | 9.6 kB  00:00:04
(4/15): docker-ce-edge/x86_64/other_db                                                                                                           |  62 kB  00:00:04
(5/15): docker-ce-stable/x86_64/other_db                                                                                                         |  66 kB  00:00:00
(6/15): base/7/x86_64/filelists_db                                                                                                               | 6.9 MB  00:00:05
(7/15): epel/x86_64/filelists_db                                                                                                                 |  10 MB  00:00:01
(8/15): epel/x86_64/prestodelta                                                                                                                  | 2.8 kB  00:00:00
(9/15): epel/x86_64/other_db                                                                                                                     | 3.1 MB  00:00:01
(10/15): extras/7/x86_64/prestodelta                                                                                                             |  48 kB  00:00:02
(11/15): extras/7/x86_64/other_db                                                                                                                |  95 kB  00:00:02
(12/15): extras/7/x86_64/filelists_db                                                                                                            | 519 kB  00:00:02
(13/15): updates/7/x86_64/filelists_db                                                                                                           | 1.3 MB  00:00:02
(14/15): updates/7/x86_64/prestodelta                                                                                                            | 231 kB  00:00:00
(15/15): updates/7/x86_64/other_db                                                                                                               | 228 kB  00:00:00
Loading mirror speeds from cached hostfile
 * base: mirror.genesisadaptive.com
 * epel: s3-mirror-us-east-1.fedoraproject.org
 * extras: mirror.math.princeton.edu
 * updates: mirror.metrocast.net
Metadata Cache Created
+ sh -c 'yum install -y -q docker-ce'
If you would like to use Docker as a non-root user, you should now consider
adding your user to the "docker" group with something like:

  sudo usermod -aG docker your-user

Remember that you will have to log out and back in for this to take effect!

WARNING: Adding a user to the "docker" group will grant the ability to run
         containers which can be used to obtain root privileges on the
         docker host.
         Refer to https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
         for more information.

If you observe the above output then you will get to know that script will detect your OS and will download, configure, and use supported repo to install Docker on your machine. It also notifies you to add non-root user to group docker so that he/she can run docker commands with root privileges.

You can download and run the script this in a single command as well like below –

root@kerneltalks # curl -fsSL get.docker.com -o - get-docker.sh | bash -s

If you are running the script on un-supported Linux version (which is not mentioned in SUPPORT_MAP list) then you will see below error.

root@kerneltalks # sh get-docker.sh
Executing docker install script, commit: 36b78b2

Either your platform is not easily detectable or is not supported by this
installer script.
Please visit the following URL for more detailed installation instructions:

https://docs.docker.com/engine/installation/

If you are on RHEL, SLES (basically Enterprise Linux editions) then only Docker EE i.e. Enterprise Edition (paid) is supported on them. You will need to purchase appropriate subscriptions to use them. You will see below message –

# sh get-docker.sh
# Executing docker install script, commit: 36b78b2


  WARNING: rhel is now only supported by Docker EE
           Check https://store.docker.com for information on Docker EE

Install with help from docker store

If both above methods are not suitable for you then you can always opt for the last method. Head to Docker online store. Goto Docker CE i.e. Community Edition (the free one) and choose your Linux distro. Currently, they have listed AWS, Azure, Fedora, CentOS, Ubuntu & Debian. Click on your choice, head to Resources tab, and click Detailed installation instructions. You will be redirected to appropriate documents on Docker documents which have detailed step by step commands to perform a clean install of Docker on Linux of your choice! Or you can always head to this home page of installation and choose your host.

Check if Docker is installed

Finally, you have to check if Docker is installed on the system. To check if docker is installed, simply run the command docker version

root@kerneltalks # docker version
Client:
 Version:      18.05.0-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   f150324
 Built:        Wed May  9 22:14:54 2018
 OS/Arch:      linux/amd64
 Experimental: false
 Orchestrator: swarm
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

The last line in the above output shows that the Docker service is not yet running on the server. You can start the service and then the output will show your Docker server details as well.

root@kerneltalks # service docker start
root@kerneltalks # docker version
Client:
 Version:      18.05.0-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   f150324
 Built:        Wed May  9 22:14:54 2018
 OS/Arch:      linux/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.05.0-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.5
  Git commit:   f150324
  Built:        Wed May  9 22:18:36 2018
  OS/Arch:      linux/amd64
  Experimental: false

So, now you have successfully installed Docker on your machine and started the Docker server. You are yet to create containers in it!

Setting up docker for non-root user

For non-root user to use Docker, you need to add the user into a group called docker. This group is automatically gets created when you install Docker.

root@kerneltalks # usermod -aG <user> docker

Run above command to add non-root user in docker group and then that user will be able to run all docker commands without root privileges.

Also, you need to make sure that docker services start automatically when the server reboots. Since system control systemctl is becoming standard on all latest Linux versions, below command will suit on nearly major Linux distros

root@kerneltalks # systemctl enable docker

This command will enable docker to run with system boot and hence no root intervention needed when the system reboots. Non-root users will continue to use docker even after a reboot.

Try Docker without installing!

If you want to try Docker without installing it on your machine then just head to Play with Docker website and you will be able to spin up machines having Docker in it. You can try Docker commands in it from your web browser!

The only limitation they have is your session will be auto closed after 4 hours. You have a clock ticking in your browser window set to 4 hours once you log in.

Execute command at shutdown and boot in Suse Linux

2 Replies

Learn how to setup commands or scripts to execute at shutdown and boot in Suse Linux

Execute a command at shutdown and boot in Suse Linux

In this article, we will walk you through the procedure to schedule scripts at shutdown and boot in Suse Linux. Many times, we have a requirement to start certain applications or services or script after server boots. Sometimes you want to stop application or service or run the script before the server shuts down. This can be done automatically by defining commands or scripts in certain files in Suse Linux.

Application auto start-stop along with OS reboot

Let’s walk through steps to configure the custom applications to auto-start and stop along with Linux reboot. Create a file with a custom name (e.g autoapp) in /etc/init.d as below –

#!/bin/sh
### BEGIN INIT INFO
# Provides: auto_app
# Required-Start: $network $syslog $remote_fs $time
# X-UnitedLinux-Should-Start:
# Required-Stop:
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Short-Description: Start and stop app with reboot
# Description: Start and stop custom application with reboot
### END INIT INFO#
case "$1" in
"start")

        su - appuser -c "/app/start/command -options"
        echo "Application started"
        ;;
"stop")
        su - appuser -c "/app/stop/command -options"
        ;;
*)
        echo "Usage: $0 { start|stop }"
        exit 1
        ;;
esac
exit 0

Make sure you copy all the above text including INIT block at the beginning of the file. Edit appuser and app commands under start and stop blocks.

Set executable permission on this file.

The next step is to identify this file as a service using chkconfig. Use filename as a service name in the below command.

root@kerneltalks # chkconfig --add autoapp

Now enable it to be handeled by systemctl

root@kerneltalks # systemctl enable autoapp

And you are done. Try to start and stop the application using systemctl command to make sure your configuration is working fine. To rule out any permission issues, script entries typo, etc.

root@kerneltalks # systemctl stop autoapp
root@kerneltalks # systemctl start autoapp

If systemctl is properly starting and stopping application as expected then you are all set. Final test you can do by rebooting your server and then verifying if the application was down while the server was shut and did it came up along with server boot.

Run script or command after server boot

In Suse Linux, you have to define commands or scripts in /etc/init.d/after.local to run them after server boots. I am running SLES 12 SP3 and my /etc/init.d/after.locallooks likes below –

root@kerneltalks # cat  /etc/init.d/after.local
#! /bin/sh
#
# Copyright (c) 2010 SuSE LINUX Products GmbH, Germany.  All rights reserved.
#
# Author: Werner Fink, 2010
#
# /etc/init.d/after.local
#
# script with local commands to be executed from init after all scripts
# of a runlevel have been executed.
#
# Here you should add things, that should happen directly after
# runlevel has been reached.
#

I added below command at end of this file.

echo "I love KernelTalks"

Then to test it, I rebooted the machine. After reboot, since command output is printed to console I need to check logs to confirm if the command executed successfully.

You can check logs of after local service as below :

# systemctl status after-local -l
● after-local.service - /etc/init.d/after.local Compatibility
   Loaded: loaded (/usr/lib/systemd/system/after-local.service; static; vendor preset: disabled)
   Active: active (exited) since Thu 2018-05-24 03:52:14 UTC; 7min ago
  Process: 2860 ExecStart=/etc/init.d/after.local (code=exited, status=0/SUCCESS)
 Main PID: 2860 (code=exited, status=0/SUCCESS)

May 24 03:52:14 kerneltalks systemd[1]: Started /etc/init.d/after.local Compatibility.
May 24 03:52:15 kerneltalks after.local[2860]: I love KernelTalks

If you observe the above output, the last line shows the output of our command which we configured in /etc/init.d/after.local! Alternatively, you can check syslog /var/log/messages file as well to check the same logs.

So it was a successful run.

Run script or command before server shutdown

To run a script or command before server initiate shutdown, you need to specify them in /etc/init.d/halt.local. Typical vanilla /etc/init.d/halt.local looks like below –

root@kerneltalks #  cat /etc/init.d/halt.local
#! /bin/sh
#
# Copyright (c) 2002 SuSE Linux AG Nuernberg, Germany.  All rights reserved.
#
# Author: Werner Fink, 1998
#         Burchard Steinbild, 1998
#
# /etc/init.d/halt.local
#
# script with local commands to be executed from init on system shutdown
#
# Here you should add things, that should happen directly before shuting
# down.
#

I added below command at end of this file.

echo "I love KernelTalks"

To make sure, this file is picked up for execution before the shutdown halt.local service should be running. Check if service is running and if not then start it.

# systemctl enable halt.local
halt.local.service is not a native service, redirecting to systemd-sysv-install
Executing /usr/lib/systemd/systemd-sysv-install enable halt.local
# systemctl start halt.local
# systemctl status halt.local
● halt.local.service
   Loaded: loaded (/etc/init.d/halt.local; bad; vendor preset: disabled)
   Active: active (exited) since Thu 2018-05-24 04:20:18 UTC; 11s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 3074 ExecStart=/etc/init.d/halt.local start (code=exited, status=0/SUCCESS)

May 24 04:20:18 kerneltalks systemd[1]: Starting halt.local.service...

Then to test it, I shut down the machine. After boot, check logs to confirm if a command was run when the system was shut down.

# cat /var/log/messages |grep halt
2018-05-24T04:21:12.657033+00:00 kerneltalks systemd[1]: Starting halt.local.service...
2018-05-24T04:21:12.657066+00:00 kerneltalks halt.local[832]: I Love KernelTalks
2018-05-24T04:21:12.657080+00:00 kerneltalks systemd[1]: Started halt.local.service.

# systemctl status halt.local -l
● halt.local.service
   Loaded: loaded (/etc/init.d/halt.local; bad; vendor preset: disabled)
   Active: active (exited) since Thu 2018-05-24 04:21:12 UTC; 1min 18s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 832 ExecStart=/etc/init.d/halt.local start (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 512)

May 24 04:21:12 kerneltalks systemd[1]: Starting halt.local.service...
May 24 04:21:12 kerneltalks halt.local[832]: I Love KernelTalks
May 24 04:21:12 kerneltalks systemd[1]: Started halt.local.service.

That’s it. You can see our echo message is printed in logs which indicates commands successfully ran before shutdown.

In this way, you can configure your application start-stop commands in Suse Linux to start and stop application after boot and before the shutdown of the server. Also, you can schedule scripts to execute before shutdown and after boot of the Suse Linux server.

How to resolve setenv: command not found

Leave a reply

setenv is a built-in command for csh. You need to have C Shell to tackle with setenv: command not found error.

Error :

Set environment command setenv is not available on the system. You see below error :

root@kerneltalks # setenv
-bash: setenv: command not found

So question is how to install setenv command.

Solution :

setenv is a shell built-in command comes with C shell csh. Above error could be due to two things –

csh is not installed on server
User havnt invoked csh shell

For point 1, go ahead and install csh package.

For point 2, Simply invoke csh shell by changing user login shell (usermod -s) or use chsh command as below –

root@kerneltalks # chsh root
Changing shell for root.
New shell [/bin/bash]: /bin/csh
Shell changed.

And to change shell on the fly for your current logged-in session use below command –

root@kerneltalks # echo $0
bash
root@kerneltalks # csh
root@kerneltalks # echo $0
csh

Now, after csh shell availability if you run setenv, it runs smooth!

# setenv
REMOTEHOST=210.23.23.456
XDG_SESSION_ID=1
HOSTNAME=kerneltalks
HOST=kerneltalks
TERM=xterm
SHELL=/bin/bash
HISTSIZE=1000
GROUP=root
USER=root
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
HOSTTYPE=x86_64-linux
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
MAIL=/var/spool/mail/root
PWD=/root
LANG=en_US.UTF-8
HISTCONTROL=ignoredups
HOME=/root
SHLVL=6
OSTYPE=linux
VENDOR=unknown
MACHTYPE=x86_64
LOGNAME=root
LESSOPEN=||/usr/bin/lesspipe.sh %s
_=/bin/csh

Why ps output shows UID instead of username

Leave a reply

Learn why ps output shows UID instead of username.

PS reads UID instead of username in output

One of our reader asked me:

I see userid in place of the username in ps -ef command output, please explain.

In this article, we will see why ps output shows UID instead of username sometimes. In some recent Linux distributions like RHEL 7, it shows cropped username ending with + sign. Let’s see the reason behind ps doesn’t show username.

Normal ps -ef command output looks like below –

root       541     1  0 17:48 ?        00:00:00 /usr/sbin/NetworkManager --no-da
root       559   541  0 17:48 ?        00:00:00 /sbin/dhclient -d -q -sf /usr/li
root       791     1  0 17:48 ?        00:00:00 /usr/bin/python -Es /usr/sbin/tu
root      1067     1  0 17:48 ?        00:00:00 /usr/libexec/postfix/master -w

where the first column is username who owns that particular process. Sometimes you see output like below –

kernelt+  1354  1335  0 17:50 pts/0    00:00:00 top
OR
1001  1354  1335  0 17:50 pts/0    00:00:00 top

where username in ps output is numeric or cropped username ending with +

This is because ps -ef output restricts username up to 8 characters. If your username is longer than 8 characters then it will display UID or cropped version of it. Here we have kerneltalks user on our server.

# cat /etc/passwd |grep kernel
kerneltalks:x:1001:1001::/home/kerneltalks:/bin/bash

If you observe, user kerneltalks has UID 1001 and hence we could see that UID in ps -ef output.

How to change UID or GID safely in Linux

4 Replies

Learn how to change UID or GID safely in Linux. Also, know how to switch UID between two users and GID between two groups without impacting files ownership they own.

In this article, we will walk you through to change UID or GID of existing users or groups without affecting file ownership owned by them. Later, we also explained how to switch GID between two groups and how to switch UID between two users on the system without affecting file ownership owned by them.

Let’s start with changing UID or GID on the system.

Current scenario :

User shrikant with UID 1001
Group sysadmin with GID 2001

Expected scenario :

User shrikant with UID 3001
Group sysadmin with GID 4001

Changing GID and UID is simple using usermod or groupmod command, but you have to keep in mind that after changing UID or GID you need to change ownership of all files owned by them manually since file ownership is known to the kernel by GID and UID, not by username.

The procedure will be –

Change UID or GID as below :

root@kerneltalks # usermod -u 3001 shrikant
root@kerneltalks # groupmod -g 4001 sysadmin

Now, search and change all file’s ownership owned by this user or group with for loop

root@kerneltalks # for i in `find / -user 1001`; do chown 3001 $i; done
root@kerneltalks # for i in `find / -group 2001`; do chgrp 4001 $i; done
OR
root@kerneltalks # find / -user 1001 -exec chown -h shrikant {} \;
root@kerneltalks # find / -group 2001 -exec chgrp -h sysadmin {} \;

That’s it. You have safely changed UID and GID on your system without affecting any file ownership owned by them!

How to switch GID of two groups

Current scenario :

Group sysadmin with GID 1111
Group oracle with GID 2222

Expected scenario :

Group sysadmin with GID 2222
Group oracle with GID 1111

In the above situation, we need to use one intermediate GID which is currently not in use on your system. Check /etc/group file and select one GID XXXX which is not present in a file. In our example, we take 9999 as intermediate GID.

Now, the process is simple –

Change sysadmin GID to 9999
Find and change the group of all files owned by GID 1111 to sysadmin
Change oracle GID to 1111
Find and change the group of all files owned by GID 2222 to oracle
Change sysadmin GID to 2222
Find and change the group of all files owned by GID 9999 to sysadmin

List of commands for above steps are –

root@kerneltalks # groupmod -g 9999 sysadmin
root@kerneltalks # find / -group 1111 -exec chgrp -h sysadmin {} \;
root@kerneltalks # groupmod -g 1111 oracle
root@kerneltalks # find / -group 2222 -exec chgrp -h oracle {} \;
root@kerneltalks # groupmod -g 2222 sysadmin
root@kerneltalks # find / -group 9999 -exec chgrp -h sysadmin {} \;

How to switch UID of two users

It can be done in the same way we switched GID above by using intermediate UID.