Locally hacking w/ kube scheduler behaviour

UPDATE

Actually, you can make service predicates by adding this to your config.json, rather than hardcoding it into scheduler startup.

{
  "name":"e2eLabelPodsAffinity",
  "argument":{
     "serviceAffinity":{
       "labels":["e2e"]
     }
  }
}

Earlier hack post 

I have been diving into the kubernetes scheduler lately, specifically, the ServiceAffinity predicate.

One thing I found is that not all predicates are enabled by default.

In order to enable the kube service affinity predicate, here is what I did.

Something similar would be done for any custom predicates/priorities which you might want to experiment with.

1) Edit plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go (or write your own provider).  The simplest way is to just hack it, and modify defaultPredicates().   This function will add in all the Predicates that you care about.  for example, adding

factory.RegisterFitPredicateFactory("ServiceAffinity", func(args factor.PluginFactorArgs alg.FitPredicate { return predicates.NewServiceAffinityPredicate(...) }

The ... is easy to fill in based on the other examples in this function.

2)   Next, you need to create a schdeuler policy json.  You can use the one in examples/ and simply add your predicate / priority to it.  i.e.

{"name":"ServiceAffinity"}

3) To test this all locally, you can use our favorite local-up-cluster.sh script.  Modify the start_kubeproxy function to add the following line under "hyperkube scheduler"...

--policy-config-file="examples/scheduler-policy-config.json"

Then you should see you're fancy new predicate being added in the startup of the /tmp/ scheduler logs.

"Registering predicate: e2eLabelPodsAffinity"
...
and so on