Shimmer Grant Reviewer Test challenge

Governance Proposals Community Treasury - Grant Reviewer applications
1

Shimmer Grant Reviewer Test challenge

To give the community a better view of the capability of the applicants of the Shimmer Grant Committee to make educated decisions on Grant proposals, I developed a little challenge.

I searched for Grant proposals in other Projects that would fit and could be submitted precisely like that to the Shimmer Grant committee. I found a lot of good examples in the Polkadot Treasury proposal program and selected two proposals for this purpose.

Proposal 1: A very technical focussed proposal for Node APIs, Key management systems, and Infrastructure that requires an excellent technical understanding of DLT Protocols to review.

This is the original Proposal submitted to Polkadot and approved in 2022 by the Polkadot treasury committee for a Grant of 60000 USD: Crypto APIs Polkadot Spending Proposal.

Proposal 2: A more community and Builder/Event focussed Proposal for a Hackathon that does not require deep technical knowledge of the Protocol as a developer.

This is the original Proposal submitted to Polkadot and approved in 2022 by the Polkadot treasury committee for a Grant of 90000 USD:

EasyA x Polkadot - Hackathon Proposal for Discussion

The Challenge:

I took those two original proposals and modified them:

  • Removed all traces that would lead readers to figure out it was submitted to Polkadot and replaced all Polkadot Mentions with the respective Shimmer wordings.
  • Created three versions of each Proposal:
    • An underfunded version
    • An overpriced version
    • The original version
    • Price changes have been established by altering the requested funding for certain project phases or reducing/increasing the daily cost basis in the project plans.

I asked our Grant reviewer candidates if they would agree to take part in this challenge on 19 November with this message sent out to everyone:

Hey, I want to challenge all potential Committee members with a little task.

I want to present you with a grant application and give you one week to come up with an opinion about this application based on the information provided to you. Please put anything that comes to your mind in your response to this message, and if possible, come up with an initial opinion if you would support funding this Proposal.

You may want to use the evaluation matrix developed for the Treasury committee: Shimmer Community Treasury Grant Committee - Version 2

The Proposal is a copy of an original proposal submitted to another Ecosystem Grant program.

I have chosen two types of applications, one that is a bit more developer oriented for the reviewers with experience in software projects and one that is more community/event focussed for the others.

Every Proposal is presented in 3 different versions. I have changed some parameters in every version, so your competitors may be presented with the same Proposal with some small but important changes to the original version.

I hope you agree to this little challenge. It may be interesting for the community to see how different candidates approach this task and to which conclusions they come regarding the grant proposal.

I will DM every candidate with the same text you got here and send them their challenge privately. I hope we can keep this private until the challenge is finished in one week.

Next Sunday, at 11 am CET, I will post the information about the challenge in reply to your application post and will include the response you sent me via DM in this post.

This will, in my opinion, be the fairest process to give everyone the same conditions without revealing a challenge that a competitor has to solve or making answers public that others could consider in their own approach to the problem. So everything will stay with me, and I am the only one who knows who gets which challenge and who replies what until the reveal.

I try to make sure that applicants who are part of the same project do not get the same grant challenge, so they cannot support each other.

Please reply within 24 hours if you agree to this challenge, and I will send you the Proposal.

Thanks again for offering your skills to the community. I hope you find this a fair approach to give the community some better insights.

All reviewers that replied have then been randomly given different types of Proposals to review with this request:

Okay, here is your challenge:

Proposal

Please request access to this document.

Simple question: Would you approve the spending for this Proposal from the Shimmer Community Treasury based on the Shimmer Community Treasury Grant Committee - Version 2 1?

If Yes, why?

If No, why not?

Please send me your response until Next Sunday at 10 am CET.

I will publish it then at 11 am.

Let me know what you think. I have heard good feedback so far on the general idea. I know the process here might not be perfect, but I hope we all can learn something useful from it.

We extended the timeframe because we had some late applications, and I wanted to let everyone participate.

The Results:

image
image1368×940 89.1 KB

I gave the 4 Candidates for the Committee Lead Position both proposals to review.

All other candidates have been given proposals regarding the skillsets they have stated in their application. All responses have been sent to me in private, and I have kept them confidential till today.

The detailed answers of all candidates can be found posted in their Grant applications. We want to have it directly in the application because this is where potential voters should find all information about a Grant Reviewer.

Detailed Responses:

Grant Committee Lead Candidates:

Grant Reviewer Candidates:

The following candidates have not responded to my request or not delivered a Review:

The following Candidates have decided to withdraw their submissions and will not run for Phase 2 of the selection process:

I hope the information is found useful by the community and helps you to select the best Committee members for the Shimmer Community Treasury Committee.

17 Likes
4

Information from the moderators:

The candidate has asked to be allowed to submit this review after the details of the challenge have already been revealed and after all reviews of the other candidates have been published.
He has not seen the message in his forum DM’s that invited him to participate in this challenge sent out on 19 November.
As he did not respond to this invitation message, he was not included in the initial challenge.
Therefore, remember that his response below is based on much more available information than the other reviewers’ in their challenge had to work with.

Before submitting this review, the candidate had the ability to access all information stated in this post: Shimmer Grant Reviewer Test challenge

Especially:

  • Read all reviews of the other Grant Reviewer candidates that did a review of the same proposal
  • know that the Polkadot Treasury committee accepted this proposal for the described budget in their Grant program.

Original post:

I’ve added my review for the Grant Reviewer Test challenge below for the proposal Crypto APIs Shimmer Spending Proposal 3

As a committee it is not simply our responsibility to accept or decline proposals, but to also find potential value within them that might support ecosystem expansion, and to provide feedback to proposal creators which may present further opportunities.

After reviewing the current proposal, I have chosen to neither decline nor accept the submission but have put forward a request for resubmission based upon the following points.

Overview

  • It is evident the team have extensive experience and their participation within the IOTA and Shimmer ecosystems, as well as their established infrastructure, network, and community, could facilitate the onboarding of valuable new developers, organisations, and projects to the IOTA community.

  • As this is clearly a project proposal written for a specific blockchain, the lack of specifics relevant to Shimmer technology is evident. With a more clearly defined proposal targeting Shimmer technology there would be greater relevance in certain aspects of the proposal.

  • The development of open-source SDKs supports the community with strong tooling. This can also bring value in allowing already existing projects the simplicity of migrating to Shimmer, bringing further value from outside the community.

  • An already existing explorer for the Shimmer network makes this part of the proposal obsolete.

  • A request for a resubmission of the proposal that focuses on the wallet SDKs and targeting the specifics of Shimmer technology, with a shorter timeframe and lower budget would be more likely to be approved.

Breakdown

Team

Highly experienced, established, publicly recognised, and historically successful in their development history.

Motivation

Offering an agreeable motive that supports decentralisation and democratisation of technology, the inclusion of multiple communities in the solution, providing evidence for value of not just technology for the Shimmer community, but also exposure for the Shimmer ecosystem to other network communities.

Reputation

A very well connected and supported project team, with extensive network in the DLT and non-DLT spaces. A strong clientele base, as well as historically successful providers of open-source projects in other communities.

Proposal

Clearly this has been written to focus on a specific blockchain and not the Shimmer network, needs to focus more specifically on the technology at hand. It would be expected that research be carried out prior to the submission of a proposal, and a greater understanding of the underlying technology be presented in any further submissions.

KMS – brings important tools to the community, whilst also providing an interoperable approach for current projects using this system to migrate and connect their projects to the Shimmer ecosystem.

Public Shimmer Nodes – a beneficial tool for the community having access to free and reliable infrastructure, although this does not clearly define or represent the infrastructure for the Shimmer network. With a bit more of an understanding and insight into the design of the system for the Shimmer network, this would provide value.

Block Explorer – as a publicly accessible explorer already exists in the Shimmer ecosystem, this part of the project is obsolete, and again clearly written for a blockchain, and not Shimmer technology.

Milestones

The initial milestones and timeline were acceptable, but with a change in project scope it is recommended to reduce the scope to the majority of content in phase one, and the relevant few sections of phase two. Redefining this into two separate milestones with clear objectives, rewritten to support the alterations highlighted in this review.

Payment

It is hard to assess the full viability of the grant payment without access to a full breakdown of costs. A more detailed overview of the expenses would help here. With a reduction in the timeline and product scope, a reduction in the resource allocation would be expected in a further submission.

Reviewer Matrix Tally
Relevance 1
Plan & Funding 4
Execution 4
Team Verification 4
Originality 2
------------------------------------ ----
Overall project score 15
5 Likes