I backup (clone) all my gists

I’ve had a bee in my bonnet for some weeks, and this is the first part in satisfying the buzzing. I’m going to backup all my Gists. I don’t make very frequent use of gists, but I have about 330 at the moment, and there are some I want to keep.

On GitHub I create a “personal access token” which is a 40-character string of hex bytes. I then install PyGithub and run the following Python3 program:

#!/usr/bin/env python -B

from github import Github	# pip install PyGithub
import json
import os

all = []

g = Github(open(os.path.expanduser("~/.gist")).read())

for gist in g.get_user().get_gists():
    all.append({
        "id"            : gist.id,
        "description"   : gist.description,
        "public"        : gist.public,
        "clone"         : gist.git_pull_url,
        "updated"       : gist.updated_at.isoformat(),
        "url"           : gist.url,
    })

    # yuck
    os.system("git clone '{0}' repos/{1}".format(gist.git_pull_url, gist.id))

with open("index.json", "w") as f:
    f.write(json.dumps(all, indent=4) + "\n")

The program obtains, via the GitHub API, a list of all gists (public and secret) and trivially clones each of the gist repositories into a directory called repos/. (Did you know a gist is in fact a git repository?)

$ ./all-gists.py
Cloning into 'repos/3017651'...
remote: Enumerating objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 3
Unpacking objects: 100% (3/3), done.
...

$ ls -la repos/3017651
drwxr-xr-x   13 jpm  staff    416 Apr  3 19:00 .git
-rw-r--r--    1 jpm  staff   1138 Apr  3 19:00 ds-babble.pl

What the program also does is to dump a file called index.json which contains an array of objects describing what I got. Therein the identifier, a description (if I originally set one when creating the gist), whether the gist is public (or secret if false) and date of last update.

[
  {
    "id": "3017651",
    "description": "Can you pronounce your DNSSEC DS record? ;-)",
    "public": true,
    "clone": "https://gist.github.com/3017651.git",
    "updated": "2015-10-06T15:58:13",
    "url": "https://api.github.com/gists/3017651"
  },
]

That way I can grep or jq my way through the index and find the directory into which I cloned the gist.

One thing to remember: a secret (i.e. not public) gist just means it’s not discoverable; if you give me its URL, I can access it.

In the 2nd part, I clone all GitHub repositories I’ve starred.

git, GitHub, and gist :: 03 Apr 2019 :: e-mail

Jan-Piet Mens

I backup (clone) all my gists