infra/codestorage/hg-ssh

Paddy 2015-10-14 Parent:2f4a2a20ad6d Child:bc0c83d5015d

2:121585c71fd7 Browse Files

Update to broadcast pushes to all our web frontends. We'll need dig, so install that when we apt-get. Fix a typo in the hashbang line for create_user.sh Add an hgrc file that goes in /etc/mercurial/hgrc to add the changegroup.fe_publish hook to all our repos. Basically, any time we get a change on disk, that automatically gets propagated out to all the frontend using our post-commit-broadcast.sh script. Write the aforementioned post-commit-broadcast.sh script. This basically finds the repo we're in (by stripping known prefixes), then uses dig to compile a list of web frontends. Finally, for each web frontend, we do an hg push over http. Not so hard, but it means all our web frontends are kept recent. This has a few shortcomings. I don't think it will work when starting a new repo; I think we need to wait for hg-repo-sync to back that up, then the web frontend to pull from backups. Another possibility is that we push, then stand up a new front end before the push makes it into the backups. The frontend then won't have that push until it pulls again from backups. These are rare, minutes-long windows where we're out of sync, so I'm really ok with these failure modes.

Dockerfile create_user.sh hgrc post-commit-broadcast.sh 

     1.1 --- a/Dockerfile	Sun Oct 11 17:06:15 2015 -0700
     1.2 +++ b/Dockerfile	Wed Oct 14 20:55:18 2015 -0700
     1.3 @@ -7,12 +7,16 @@
     1.4  RUN chmod +x /usr/local/bin/helpers/run-ssh.sh
     1.5  ADD pullkeys.sh /usr/local/bin/helpers/pullkeys.sh
     1.6  RUN chmod +x /usr/local/bin/helpers/pullkeys.sh
     1.7 +ADD post-commit-broadcast.sh /usr/local/bin/helpers/broadcast-to-frontends.sh
     1.8 +RUN chmod +x /usr/local/bin/helpers/broadcast-to-frontends.sh
     1.9 +
    1.10 +ADD hgrc /etc/mercurial/hgrc
    1.11  
    1.12  RUN mkdir /var/run/sshd
    1.13  
    1.14  # install required packages
    1.15  RUN apt-get -y update
    1.16 -RUN apt-get -y install openssh-server 
    1.17 +RUN apt-get -y install openssh-server dnsutils 
    1.18  
    1.19  #ADD sshd_config /etc/ssh/sshd_config
    1.20  RUN sed -ri 's/session    required     pam_loginuid.so/session    optional     pam_loginuid.so/g' /etc/pam.d/sshd

     2.1 --- a/create_user.sh	Sun Oct 11 17:06:15 2015 -0700
     2.2 +++ b/create_user.sh	Wed Oct 14 20:55:18 2015 -0700
     2.3 @@ -1,4 +1,4 @@
     2.4 -#/bin/bash
     2.5 +#!/bin/bash
     2.6  
     2.7  USERNAME=$1
     2.8  IDS=$2

     3.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     3.2 +++ b/hgrc	Wed Oct 14 20:55:18 2015 -0700
     3.3 @@ -0,0 +1,2 @@
     3.4 +[hooks]
     3.5 +changegroup.fe_publish = /usr/local/bin/helpers/broadcast-to-frontends.sh

     4.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     4.2 +++ b/post-commit-broadcast.sh	Wed Oct 14 20:55:18 2015 -0700
     4.3 @@ -0,0 +1,24 @@
     4.4 +#!/bin/bash
     4.5 +
     4.6 +cwd=`/bin/pwd`
     4.7 +me=`whoami`
     4.8 +stripped=${cwd#/home/$me/repos/}
     4.9 +
    4.10 +if [ $stripped = $cwd ]
    4.11 +then
    4.12 +	stripped=${cwd#/mounted/repos/}
    4.13 +fi
    4.14 +
    4.15 +if [ $stripped = $cwd ]
    4.16 +then
    4.17 +	echo "Can't get repo name from ${cwd} aborting"
    4.18 +	exit 0
    4.19 +fi
    4.20 +
    4.21 +FRONTENDS=$(dig mercurial-uwsgi-headless.default.svc.cluster.local +short)
    4.22 +for fe in $FRONTENDS
    4.23 +do
    4.24 +	hg push http://${fe}:8080/${stripped}
    4.25 +done
    4.26 +
    4.27 +exit 0