Approach comparison methodology
Methodology behind how approaches are being compared and analysed across the documentation.
A number of analytical comparisons are made across this documentation. These comparisons explore the trade offs between approaches that could be used for different parts of a treasury system. All of these comparisons start by listing out the most important factors that can be considered when analysing the differences between any of the approaches. Each approach is then analysed against those factors to see how effective it is at addressing them.
Factor information
Factors in any of the comparisons will include the following information:
Factor name - A short title and name to describe the factor for consideration.
Description - An overview of what the factor is about and why it is relevant.
Maximum score - There are two scores given that are out of 5 that create a final score for each approach. The first score is the maximum score for the factor. The maximum score highlights how important the factor itself is for the approaches comparison. The maximum score ranges from 1 to 5. When an approach is being analysed a second score is given to that approach that highlights how effective that approach is at addressing the factor. The score received by the approach is then applied to the maximum score to create a final score. Doing this means that the most important factors will have more weighting in the total score when compared to other factors that might be less important. Consider the following examples:
Maximum score is 5. This means the factor is very important. Here are the final scores an approach could receive depending on how well it scores in addressing this factor:
1 points out of 5 ⇒ 20% of 5 maximum score ⇒ 1
2 points out of 5 ⇒ 40% of 5 maximum score ⇒ 2
3 points out of 5 ⇒ 60% of 5 maximum score ⇒ 3
4 points out of 5 ⇒ 80% of 5 maximum score ⇒ 4
5 points out of 5 ⇒ 100% of 5 maximum score ⇒ 5
Maximum score is 4. This means the factor is fairly important. Here are the final scores an approach could receive depending on how well it scores in addressing this factor:
1 points out of 5 ⇒ 20% of 4 maximum score ⇒ 0.8
2 points out of 5 ⇒ 40% of 4 maximum score ⇒ 1.6
3 points out of 5 ⇒ 60% of 4 maximum score ⇒ 2.4
4 points out of 5 ⇒ 80% of 4 maximum score ⇒ 3.2
5 points out of 5 ⇒ 100% of 4 maximum score ⇒ 4
Maximum score is 3. This means the factor is moderately important. Here are the final scores an approach could receive depending on how well it scores in addressing this factor:
1 points out of 5 ⇒ 20% of 3 maximum score ⇒ 0.6
2 points out of 5 ⇒ 40% of 3 maximum score ⇒ 1.2
3 points out of 5 ⇒ 60% of 3 maximum score ⇒ 1.8
4 points out of 5 ⇒ 80% of 3 maximum score ⇒ 2.4
5 points out of 5 ⇒ 100% of 3 maximum score ⇒ 3
Scoring questions - A list of questions that help with thinking about how effective each approach is at addressing the factor or not. Answering these questions will help with identifying a suitable score to give each approach for that factor.
Advantages of this comparison approach
There are a number of reasons why this approach for making comparisons is effective:
Simple - This format makes it easy for people to read about the different factors that are important for this comparison. It also makes it easier to agree or disagree with any given parts of the analysis which can help to improve it over time wherever this is necessary.
Concise - The analysis for each approach only adds enough information to make an informed judgement on what the score should be when also considering the other approaches in the comparison. This helps to keep the analysis concise and informative. If there are multiple approaches that appear to be effective at addressing the listed factors then a more detailed piece of analysis could be done separately.
Effective for ranking - By giving each approach a score for each factor an easier comparison can be made about which approaches are likely the most effective for different areas. The aggregation of these factors helps with identifying the most promising approaches overall.
Visual outcome - Generating scores for each approach and each factor means a visual table can be produced for every comparison. This makes it much quicker to find out what the outcome is for any comparison without needing to read the full content.
Identifies experiment opportunities - These comparisons are an effective way to identify the approaches that could be highly effective for treasury systems. These approaches can then be compared with what is actually happening across the industry and help with highlighting the most promising experiments for Web3 ecosystems to explore.
Last updated