Leveraging the Docker API inside [ or outside ] your containers with or without sockets.

Lately I've been having to build out infrastructure to handle  container events, for example, when images are created in an openshift cluster, caching them locally inside a container for scanning purposes, and so on.    It turns out that you *really need* a docker daemon to be mounted to do this, because the docker client can't pull images (remember --- docker images are a series of layers, that can quickly be composed, so pulling one down isn't necessarily a trivial REST request).  

Lets say, you've decided, you need to do operations directly against your docker API.  For example, you want to locally pull down every image and do a running calculation of your organizations average image size.  Or, maybe you want to see how many images are created a minute, or what the average pull time is for images.... 

So, first off you need to decide where your daemon is: Yes you (pretty much) need a daemon.

- If you have control over your docker infrastructure, you can relaunch docker daemons to *bind* to an externally accessible docker IP address.

- If you *dont* have control over your docker infrastructure, then you'll very likely need to hijack the docker socket that is already running.  This means directly referencing it as the endpoint: (i.e. create a connection to a socket, and then curl from localhost using that same connection). If your doing things in containers, it  also means mounting a socket from a docker daemon into your container.... 

- Of course alternatively to hijacking docker Damon's socket, OR launching docker so that you didn't need to talk over the internal socket, you could possibly do the third option: run a brand new docker daemon inside a container, or start a brand new docker daemon on a new VM...  but that would be really really weird, mainly because, if you can run Docker-in-docker, then you probably are in the 1st category above, b/c your able to mount a bunch of system directories (IIRC , this is more then just the docker socket) and run things in privileged mode, which means you don't need to be reading this paragraph anyway.

- Oh, wait, theres a fourth alternative, use the docker curl shell script thats posted up on GitHub.  However, my friend Matt fennwick mentioned that its completely coupled to dockerhub, and isn't readily hackable.

Anyways, so, how do I start doing docker meta stuff now that I picked a daemon.

Ok, well , I'll assume you picked talking over the socket, since thats the easiest and doesn't require starting docker over in most default situations, since usually, everyone just runs the daemon on the socket without blinding to IP.... 

.... Note that the easiest way to do this is via curl commands.  Well, try this: 

 curl --unix-socket /var/run/docker.sock -X POST http://localhost/images/create?fromImage=centos7

yay

Now to see the other commands you can play with:

https://docs.docker.com/engine/api/v1.24/

There are also plenty of docker API clients you can use, but for basic functionality like Pulling images, getting images, or even saving those images as tarballs, you don't need any kind of language specific API client.  

1. Read https://nathanleclaire.com/blog/2015/11/12/using-curl-and-the-unix-socket-to-talk-to-the-docker-api/ and 
2. The simplest way to directly talk to docker over REST, is to use your socket: curl --unix-socket /var/run/docker.sock http://localhost/…. 
3. In openshift, kubernetes, etc, you’ll want to volume mount the above socket, so that your container can access the host daemon.
4. To test your containerized code outside of docker, you can easily just run something that locally talks to the docker socket, and use the below command to see what requests are going through.
5. From a dev perspective, you can debug API requests like this: curl --unix-socket /var/run/docker.sock http://localhost/events
6. Make sure you close your response bodies !  IF you don’t, POST requests and so on may never result in daemon actions.  Again, use the above /events/ endpoint to debug wether your commands are working properly.

... See the docker website for examples., i.e. https://docs.docker.com/develop/sdk/examples/#run-a-container...

Once you've got all this working, you can wrap it in your favorite language, or just consume the regular API clients (its easier once you DIY to use them properly).  Again *make sure you close your response bodies* if you build your own tooling.  Otherwise POST's and so on will hang.