Scoring calculation

Proposals Pitches
1

Proposal one-liner: Calculate the average score based on the scores in multiple fields – in the form and sub-form.

**Value proposition:**If the risk level of the case is determined by calculating the scores from assessment questions, this would help automate the risk level and avoid possible inaccuracy in the manual calculation.

Brief explanationThere are multiple score questions (the score is 1-4) in multiple sub-forms. The number of score questions are different in different sub-forms. E.g., there are 3 score questions in 1 sub-form and 5 in another.
We need to calculate the average score i.e calculate the sum of the scores (multiple questions) and divide by the number of scored questions. For eg: if there are 5 score questions, sum the score from those 5 questions and divide by 5.

Acceptance criteria

Time estimation

Risk estimation

Proposal Stakeholders

2

Something kind of similar exists in the GBV module but this only works in one specific sub form and also because all the field ID’s are fixed in GBV.
To have this working in a generic manner would mean a new field type that required you to select other fields (potentially across multiple forms and sub forms) and then also the aggregation function you require.
I think this is non trivial to implement

3

So @Ian, are you saying that having this feature would mean we’d need to have to “lock down” the configuration and limit the functionality to preset fields?

4

No. If it were to be implemented as it is in GBV then it would mean that. Of course that is not realistic with CP. This means that we need to make it generic (it works with one to many numerical fields across forms and sub forms) and we need to provide a UI to allow users the ability to choose those fields and the mathematical function to be applied on them. Nothing like that exists in the forms currently - the nearest thing is the reports interface I think. We are lucky that Primero already understands some of the backend, for example what ‘calculation’ and ‘avg’ mean

Field.new('name' => 'pss_score',
            'type' => 'numeric_field',
            'calculation' => {
              'type' => 'number',
              'expression' => {
                'avg' => [ 'giving_advice', 'exchanging_ideas', 'uniting_with_other_community',
                  'asking_getting_help', 'making_important_decisions',
                  'taking_part_in_family_decisions', 'learning_new_skills',
                  'concentrating_on_your_tasks', 'interacting_dealing_people',
                  'keeping_household_clean' ]
              }
            },

however what seems at first glance to be a small self contained piece of work is actually a little more involved than that

5

Hi! I just wanted to add that for this automated calculation effort, it’s essential to include decimals. If you round up, the nuance needed for averages is lost. This is something we have really needed, too! Specifically, we need an automatic calculation of average scores that go to the hundredths place. For example: 2/3 = 0.67

6

Thanks Megan, it’s helpful to hear that your team also needs this. Is there a specific set of questions that we can model this after?

7

Hi Robert! We had hired Jozian to create a tallying function that automatically calculates scores for two surveys - the Felt Stigma and Psychosocial Wellbeing subforms. These surveys measure the outcome and impact that case management services have on survivors to better inform their individual case management services, as well as broader programmatic decision making.

To calculate the scores, you add the points across all 10 items, then divide the total by 10 (the total number of questions). If the survivor skipped one or more items in the questionnaire, their average point score is calculated for the answered questions only. To do this, the points for all answered questions are added together, and divided by the total number of questions answered. For example, if the survivor answered 8 questions, the sum is divided by 8.

The issue is, the final automatic calculation rounds the average to a whole number, without decimals. The entire scoring system, however, is developed and based on decimals to the hundredth place. Without the decimals, these automatic calculations are unusable.

The good news is the coding is there! Automatic calculations exist! We just need to include decimals and not have them round to whole numbers. Let me know if this is unclear - I’m happy to explain further.

8

Based on the discussion here is a scope for the work:

  1. Provide a calculated field type and expose this calculated field type in the UI
  2. Ensure that this field type supports aggregations
  3. Provide an interface to select 1-n numerical fields across forms and subforms (this should be limited to within discreet modules I assume?)
  4. Modify the avg function to round to some arbitrary number of decimal places
  5. Output the result of the calculation into a read only (disabled=true) field. Note that support for outputting the result into labels such as question help texts for example is out of scope
  6. Run QA and write tests for the modified functionality

Time: 2 months
Estimate: 5 ETH

9

Hi Ian. I think the approach sounds good. To your question on point 3 (limiting this to discreet modules), I think that’s correct but I’d like to get more clarity. I’d also like to request that you submit the cost estimate in USD as the ETH value has been fluctuating a lot

10

Point 3 just to clarify:
We have an instance with both CP and GBV modules deployed. The same questions are asked in both modules about risk. A case suffered GBV and requires CP
We should NOT be able to work out an average in this case across modules. We can work out an average risk for the child for CP and then a separte average risk for GBV - but we cannot work out a combined average risk
On todays eth price the 5 eth estimate for the work is $US18k in dollars

11

Thanks for clarifying. AS defined in your approach, this seems like it might be slightly too large a scope of work for the DAO. Can you suggest ways we might chunk it out into smaller pieces?

12

Probably the easiest way would be to separate the front end and back end work. Back end work would be the calculated field type, aggregations, mathematical functions, qa etc. It would be slightly less than 50% of the estimated amount. Front end work would be exposing fields in the UI, selecting fields in forms and sub forms, outputting results coherently and writing tests/qa. This would be the rest of the amount.
Problems with this are that the backend work is not really visible to end users so the acceptance criteria is harder to establish. Also the front end work will take longer to do (with much more back and forth) than the backend stuff

13

Hi Ian, I think the perhaps best way forward is to take on these 2 specific projects (Phanneth’s request from Cambodia and Megan’s for GBVIMS+). From this we could learn more about how we might can make these features available (through the form builder) in a future release. If we took that approach, how would your level of effort estimate change?

14

The work then would be:

  1. Modifying the calculation function to enable decimals for GBV folks
  2. Verifying the GBV calculation work is recognised on CP and porting over whatever is missing or does not work
  3. Configuring the server for @pkhauv to enable calculations. This will just be in a single form or a single sub form for now with values hardcoded

Estimate: US5k

15

I think this sounds like a better approach. @megan.obrien what do you say?

16

If it’s helpful to understand what we are looking for, here is the scope:

  • Create a tallying function that adds the points from the responses entered into Felt Stigma Subforms

  • Create a tallying function that adds the points from the responses entered into Psychosocial Wellbeing Subforms

  • Create a tallying function that provides a score for each Felt Stigma Subform. This is calculated adding the points across all 10 items, then dividing the total by 10. If the survivor skipped one or more items in the questionnaire, their average point score is calculated for the answered questions only. To do this, the points for all answered questions are added together, and divided by the total number of questions answered. For example, if the survivor answered 8 questions, the sum is divided by 8. This will require decimal points to be available as a part of number responses.

  • Create a tallying function that provides a score for each Psychosocial Wellbeing Subform. This is calculated adding the points across all 10 items, then dividing the total by 10. If the survivor skipped one or more items in the questionnaire, their average point score is calculated for the answered questions only. To do this, the points for all answered questions are added together, and divided by the total number of questions answered. For example, if the survivor answered 8 questions, the sum is divided by 8. This will require decimal points to be available as a part of number responses.

  • Ensure that the scores are included when the Felt Stigma and/or Psychosocial Wellbeing subforms are exported

I’m happy to jump on a call to discuss further if that would be useful!

17

I think we are ready to vote on this one. What do others think?

1 Like
18

I am happy to vote - definitely in favor of getting the above completed!

19

Here is a recap of the updated proposal given the feedback:

Proposal one-liner:
Decimals on scores and configuration of a server to enable them per unique form

Value proposition
Automate some specified calculations and avoid possible inaccuracies when using manual calculations.

Brief explanation
The scope of the work is as follows:

  1. Modify the calculation function to enable decimals as an output
  2. Verify that the current calculation work is recognised on the child protection module and port over whatever is missing or does not work
  3. Configure the server for Cambodia to enable calculations for a single form or a single sub form. The values will be hardcoded into the server configuration and will not be user configurable

Acceptance criteria
The first acceptance is getting the decimal changes accepted into the Primero codebase (along with a test). The second will be UAT from the Cambodia team

Time estimation
Some parts of this (particularly the second) rely on factors external to the development team. Consequently we need to allow 2 months

Risk estimation:
The major risk is one of expectations from the field teams in Cambodia so we need to align on this before we start

Proposal Stakeholders
UNICEF Cambodia, Primero CP and GBV teams