If we’re allowed to use an oracle, then I would be quite interested in incorporating a PageRank-style reputation system in which community members can “endorse” other community members and these endorsements are used to generate contribution scores – an endorsement from someone who is also endorsed is worth a lot, while an endorsement from someone who is unendorsed is worth little. The subjectivity gets around the Goodhart-related problems which we get into when we use objective metrics.
It can also be made somewhat sybil-resistant by limiting who gets a contribution score “to start” (i.e. we use objective github data to determine the “starting contribution distribution” and then use the subjective endorsements to distribute the contribution scores beyond what the data themselves can capture). If someone creates a fake account, their endorsements have zero weight until someone endorses them. So not sybil-resistant per-se, but not trivially sybillable either.
I like this idea because it is grounded in something objective (i.e. github contributions) but incorporates subjective information (endorsements) beyond what the objective can provide. It also means that we don’t have to worry about capturing every objective signal as the subjective assessments will “smooth out” the gaps in what the metrics show.