Quickly get that damn GlusterFS distriubted storage working on kubernetes

Heres how to do it.  Without any kind of declarative PVC, or CNS or ... just run it in damn VMs and mounting it to a volume.

This is how we used to do it in the old days.  Apologies to Steve and Brad who probably have a hipper, easier way now - I still cant figure out anything better then the old 2014 hadoop on glusterfs implementation that we used to use, so I'm posting it here !

1) Setup 2 VMs w nothing other then centos7.

2)  yum install centos-release-gluster

Then On each:

2)  sudo mkfs.xfs -i size=512 /dev/sdb

3) echo '/dev/sdb /data/brick1 xfs defaults 1 2' >> /etc/fstab

4) mount -a && mount 

5) mkdir -p /data/brick1

6) yum install glusterfs-server -y 

7) service glusterd start

Now Peer probe the nodes across one another: 

8) For example on if the nodes hostnames are "gluster1" and "gluster2"

ssh into "gluster1", and then run gluster peer probe gluster-2

9) Confirm that the nodes are peered: 

    ssh into "gluster2", and then run gluster peer status It should say that its paired to another node, like this: 

  gluster> peer statusNumber of Peers: 1

  Hostname: gluster-1.c.gke-verification.internalUuid: 71596528-d3b5-4580-bd0c-f2fb666c0bf6

  State: Peer in Cluster (Connected).

Now, test the distributed file system:

10) gluster volume create gv0 replica 2 gluster-1:/data/brick1/gv0 gluster-2aa:/data/brick1/gv0

11) mkdir /glustermount/ && mount -t glusterfs gluster-1:/gv0 /glustermount

12) touch /glustermount/abcd 

13) ssh gluster-2 ls /glustermount

In step 13) you should see 'abcd' pop up, and then, your ready to test on a real, distributed storage system.

Now, add those glusterFS endpoints to your kube cluster !

---

apiVersion: v1

kind: Service

metadata:

  name: glusterfs-cluster 

spec:

  ports:

  - port: 1

---

apiVersion: v1

kind: Endpoints

metadata:

  name: glusterfs-cluster 

subsets:

  - addresses:

      - ip: 35.226.****** (the IP address of gluster1)

    ports:

      - port: 1

  - addresses:

      - ip: 35.225.******  (the IP address of gluster2)

    ports:

      - port: 1 

Then you can create a pod that mounts these contents like so

{

    "apiVersion": "v1",

    "kind": "Pod",

    "metadata": {

        "name": "glusterfs"

    },

    "spec": {

        "containers": [

            {

                "name": "glusterfs",

                "image": "nginx",

                "volumeMounts": [

                    {

                        "mountPath": "/mnt/glusterfs",

                        "name": "glusterfsvol"

                    }

                ]

            }

        ],

        "volumes": [

            {

                "name": "glusterfsvol",

                "glusterfs": {

                    "endpoints": "glusterfs-cluster",

                    "path": "gv0",

                    "readOnly": false

                }

            }

        ]

    }

}