Watch for changes

✓ Subscribed

SCENE SETTERS

The NordDEC begins with a series of questions to capture an app's core purpose and functionality. These include the target audience, the type of data the app collects and the apps primary functions and features. None of the scene setter questions are intended to have any scoring or risk implications and are purely to decide on the line of enquiry further in the evaluation.

DATA & PRIVACY

Privacy Policy

Initially, the evaluation identifies the relevant privacy policy for the app, which is available to users through the app and/or the App Store or Play Store and/or on the website. The more transparent the privacy policy, the better. Ultimately, the privacy policy must clearly state that user data will not be used or shared with other parties, except as described in the privacy policy, or without express consent of the user. Ideally it will identify:

·         what data is collected from the user and how,

·         if the user is informed of the developer’s intentions with processing and sharing their data, and

·         if the user’s consent is obtained.

The privacy policy should accurately reflect the data usage of the app. The Assessors will be able to note if any data is collected outside of what is detailed in the privacy policy. Additionally, the policy should inform users of the developer’s intent to use their data for marketing purposes. If user data is shared for any other purposes other than basic use of the app, or legal obligations, then the evaluation considers if the user is able to opt out of these activities.

Question

Answer Type

ORCHA Question Reference Source

Framework

Is there a Privacy Policy clearly available via the Web App/ Website? (Only relevant for Web Apps)

Yes/No

ORC_D39a

OBR

Is there a Privacy Summary published anywhere by the developer? (Only relevant to Mobile Apps)

Yes/No

ORC_D39b

OBR

Is the Privacy Policy made immediately available when the user first opens the app?

Yes/No

ORC_DP03

OBR

Is the policy made available when the user is signing up to the service?

Yes/No

ORC_DP04

OBR

Is it published within the app? 

Yes/No

ORC_DP01

OBR

Is it available externally via the app, or via a linked website?

Yes/No

ORC_DP02

OBR

Is it available via the relevant app store?

Yes/No

ORC_DP05

OBR

What data does the Privacy Policy state the developer collects? 

Multiple Choice

ORC_DP06

OBR

Is the policy accurate, with regards to the data the developer intends to collect?

Yes/No

ORC_DP07

OBR

Does the app state that data collected by the app is stored locally, unless the user manually exports the data?

Yes/No

ORC_D10a

OBR

How does the developer obtain consent for the processing of user data?

Yes/No

ORC_DP08

OBR

Does the Privacy Policy Provide the name and contact details of their Data Protection Officer (DPO), or similar individual representative for the company?

Yes/No

ORC_DP14

OBR

Provide the details of the DPO: (Text Response)

Free Text

ORC_DP15

OBR

Data use

Once it is established what data is collected by the app, the evaluation looks at how that data is used and shared, and if this is communicated to the user. The privacy policy should state all intended uses and legal basis of processing user data, such as legal obligation, research or marketing. Users should also be given the option to withdraw consent for the use of their data, particularly for marketing.

Question

Answer Type

ORCHA Question Reference Source

Framework

Does the developer fully inform the user of how they will collect data about them?

Yes/No

ORC_D69

OBR

Does the developer provide users with details on all the purposes of processing user data?

Yes/No

ORC_D13

OBR

What is automatically shared data used for?

Multiple Choice

ORC_DP10

OBR

Does the developer appear to intend to share or process the user data collected by the app for any purposes that have not been made clear to the user, or for any purposes they deem necessary?

Yes/No

ORC_D38

OBR

Does the developer inform users that they would like to use their data for the purpose of marketing?  

Yes/No

ORC_D71

OBR

Does the developer obtain informed consent separately, for the purpose of marketing?

Yes/No

ORC_DP12

OBR

Is the user informed of how they can opt out of each of these activities?

Yes/No

ORC_D28

OBR

If the user can not opt out of all processing activity, does the developer clearly explain which activities they cannot opt out of and why?

Yes/No

ORC_DP13

OBR

Is the user informed that their data will not be shared with other parties, except for the purposes that have been set out in the privacy policy?

Yes/No

ORC_D16

OBR

Data Storage and Transit/Transfer

The key areas in this section are surrounding data storage and data transfer. The data privacy policy should inform the user of where their data is stored, how their data is protected in storage, and how it is protected in transit between the user’s device and the host storage. The NordDEC looks for specific and secure storage techniques, such as encryption or firewalls. During transit, it is preferable that data is protected using SSL encryption.

Question

Answer Type

ORCHA Question Reference Source

Framework

Does the data privacy policy or equivalent provide detail about where the data collected by the app will be stored (i.e. on the app or in an external data warehouse, cloud server etc.)?

Yes/No

ORC_DST01

OBR

Where is the data stored? 

Free Text

ORC_DST02

OBR

Does the data privacy policy, or equivalent, state whether personal data is stored using recognised secure data storage technologies?

Yes/No

ORC_DST03

OBR

Is all personally identifiable data encrypted in transit between the device and any external host storage?

Yes/No

ORC_D17

OBR

Is the user informed that online video consultations use secure encryption methods?

Yes/No

ORC_DST04

OBR

Data Standards and Management

The NordDEC will award additional points if an app developer is compliant with any recognised International Data Management Standards such as ISO 27001. The privacy policy should inform users of a data retention period, and a method for data destruction. The NordDEC also identifies whether the developer has a policy in place to deal with any data security breaches.

Question

Answer Type

ORCHA Question Reference Source

Framework

Does the policy state its compliance with recognised International Data Management Standards?

Multiple Choice

ORC_DM01

OBR

Does the policy contain details of the length of time data is retained?

Yes/No

ORC_D19

OBR

Is there a statement containing details of a method for data destruction?

Yes/No

ORC_D20

OBR

Is there a statement that sets out a process for managing data confidentiality breaches?

Yes/No

ORC_D21

OBR

GDPR

This evaluation area focuses on the General Data Protection Regulation (GDPR), which in May 2018 came into force. The NordDEC is concerned that all apps, particularly those developed in the EEA, are fully compliant with the GDPR. This means a clear and explicit statement of compliance, as well as confirming that the user is entitled to the 7 user rights.

The developer should also inform the user of how they can exercise these rights, and should commit to responding within a time frame of 2 months or less. Under the GDPR, the policy should outline the legal basis for collection of user data, and ensure that only minimal data is collected from the user.

All question relating to GDPR will only be asked for apps that collect and process personal and/or sensitive data.

Question

Answer Type

ORCHA Question Reference Source

Framework

Is there a statement that confirms the App’s compliance to GDPR 2018?

Yes/No

ORC_D23

OBR

Is the user informed of the legal basis for which data is collected from them?

Yes/No

ORC_D60

OBR

Is the user informed of the developer’s intent to ensure that data minimisation principles are met?

Yes/No

ORC_DPR03

OBR

Is there a statement that the policy will be updated duly should the purpose of data collection change? This may mean re-obtaining consent (if consent was the lawful basis).

Yes/No

ORC_D61

OBR

Are users informed of their rights with regards to their data? Are users clearly informed of the individual privacy rights they are entitled to expect under GDPR?

Yes/No

ORC_DPR01

OBR

Has the developer made the existence of the data subject’s right to request that their personal data is deleted clear?

Yes/No

ORC_D93

OBR

Has the developer made the existence of the data subject’s right to access their personal data clear?

Yes/No

ORC_D25

OBR

Has the developer made the existence of the data subject’s rights to rectify their personal data clear?

Yes/No

ORC_D56

OBR

Has the developer made the existence of the data subject’s rights to restrict the use of their personal data clear?

Yes/No

ORC_D81

OBR

Has the developer made the existence of the data subject’s rights to object to the processing of their personal data clear?

Yes/No

ORC_D57

OBR

Has the developer made the existence of the data subject’s rights to portability of (receive) their personal data clear?

Yes/No

ORC_D59

OBR

Has the developer made the existence of the data subject’s right to withdraw consent for the use of their personal data clear?

Yes/No

ORC_D58

OBR

Has the developer made clear the existence of the user’s right to request that they are not subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her?

Yes/No

ORC_DPR02

OBR

Does the developer provide details which the user can contact them on to exercise their rights?

Yes/No

ORC_D82

OBR

Is the user informed of the time frame in which the developer will respond to any requests to exercise their rights?

Yes/No

ORC_D83

OBR

Other Data Questions

This section also looks into children’s data use (if applicable), or if a user can report knowledge of a child accessing the app without parental consent. The transparency of the privacy policy should extend to inform the user that any links to third party websites or apps are not covered by the developer’s privacy policy, and users should make themselves aware of such third party policies. The privacy policy should contain contact details, should the user wish to make further enquiries regarding their data. The NordDEC also awards additional value points if the app provides the user with an additional, optional layer of security to protect their data.

Question

Answer Type

ORCHA Question Reference Source

Framework

Are users clearly informed of the use of cookies when first landing on the developers site/app?

Yes/No

ORC_D99

OBR

Are user's required to confirm their acceptance of the developer's use of cookies, when initially informed of the use?

Yes/No

ORC_D100

OBR

Does the developer provide a full Cookie Policy, separate from the Terms of Service and/or Privacy Policy?

Yes/No

ORC_D84

OBR

Is the app ‘particularly likely’ to be used by children, even if they are not the primary market for the product?

Yes/No

ORC_D44

OBR

Are users informed of how they can report, to the developer, any knowledge of a child accessing the app and providing personal data, without parental consent?

Yes/No

ORC_DO01

OBR

Is the user made aware that by following links to third party websites, the developer’s policies no longer apply, and that the user should make themselves aware of the third party’s policies?

Yes/No

ORC_D91

OBR

Is the user informed of how they can make further enquiries about the company’s privacy policy?

Yes/No

ORC_D92

OBR

Does the app allow the user to set their preferences for sharing the app data with or from other apps (e.g. Facebook / Instagram/Fitbit etc)?

Yes/No

ORC_D06

OBR

Is there functionality within the app to allow the user to set their preferences for sharing app data with others users (clinicians, carer, family, friends, buddies)?

Yes/No

ORC_D27

OBR

Is it strictly necessary for anyone to easily access the personal information that persists on the app? e.g. to access health info during an emergency.

Yes/No

ORC_DO02

OBR

Are users provided options to introduce additional security measures to protect their data on the app? eg. set additional pass codes for access to the app, after accessing the device is unlocked.

Yes/No

ORC_DO03

OBR

Does the app use a sign up/sign in verification/authentication model

Yes/No

ORC_DO04

OBR

What type of model is being used? (please describe)

Multiple Option

ORC_DO05

OBR

Enhanced Data evaluation

Question

Answer Type

ORCHA Question Reference Source

Framework

Is the developer of the app the Data Controller, Data Processor or Product Manufacturer Only?

Multiple Options

ORC_ERC_EDC_O01

ERC

Is the developer is subject to EU Data Protection Laws?

Yes/No

ORC_ERC_EDC_O02

ERC

Provide EU data protection registration details:

Free Text

ORC_ERC_EDC_O03

ERC

If this information is unavailable, please explain why?

Free Text

ORC_ERC_EDC_O04

ERC

Is the developer of the App a public authority or body?

Yes/No

ORC_ERC_EDC_DPO01

ERC

Do any of the developer’s processing activities include automated systematic and extensive profiling with significant effects?

Yes/No

ORC_ERC_EDC_DPA01

DTAC

Do any of the developer’s processing activities include large scale use of sensitive/special category data?

Yes/No

ORC_ERC_EDC_DPA02

DTAC

Do any of the developer’s processing activities include systematic monitoring of a publicly accessible area on a large scale?

Yes/No

 ORC_ERC_EDC_DPA03

DTAC

Do any of the developer’s processing activities include any of the following indicators of high risk processing?

Multiple Choice

 ORC_ERC_EDC_DPA04

DTAC

Is the developer required to have a Data Protection Officer?

Yes/No

ORC_ERC_EDC_DPO06

DTAC

Is the developer required to have written a Data Protection Impact Assessment (DPIA)?

Yes/No

 ORC_ERC_EDC_DPIA01

DTAC

Has the developer submitted a DPIA or made a DPIA publicly available?

Yes/No

ORC_ERC_EDC_DPIA02

DTAC

Has the developer, at a minimum, published details on how to communicate with the DPO, for individuals to contact the DPO as needed?

Yes/No

ORC_ERC_EDC_DPO02

DTAC

Is there evidence that the DPO has the necessary professional qualities, and in particular, experience and expert knowledge of data protection law?Edit Logic 

Yes/No

ORC_ERC_EDC_DPO04

DTAC

Does the DPO hold a position within the organisation that may lead him or her to determine the purposes and the means of the processing of personal data, or require him or her to engage in further tasks and duties that may result in a conflict of interests with the primary tasks of a DPO?

Yes/No

ORC_ERC_EDC_DPO05

DTAC

If a DPIA has been submitted, does the document provide details of a Data Protection Officer?

Yes/No

ORC_ERC_EDC_DPO03

DTAC

Does the DPIA describe how the developer will collect data about individuals?

Yes/No

 ORC_ERC_EDC_DPA05

DTAC

Does the DPIA describe how the developer will store the data collected?

Yes/No

 ORC_ERC_EDC_DPA06

DTAC

Does the DPIA describe what the data collected by the developer will be used for?

Yes/No

 ORC_ERC_EDC_DPA07

DTAC

Does the DPIA provide details of who will have access to the data collected?

Yes/No

 ORC_ERC_EDC_DPA08

DTAC

Does the DPIA provide details of who the developer will share data with (any third parties)?

Yes/No

 ORC_ERC_EDC_DPA09

DTAC

Does the information regarding data sharing include details of the developer’s use of any processors?

Yes/No

 ORC_ERC_EDC_DPA10

DTAC

Does the DPIA provide details of their own data retention periods?

Yes/No

 ORC_ERC_EDC_DPA11

DTAC

Does the DPIA provide details of the length of time each third party recipient of data will retain data for?

Yes/No

 ORC_ERC_EDC_DPA12

DTAC

Does the DPIA provide details of the security measures that have been put in place in order to protect the data being processed?

Yes/No

 ORC_ERC_EDC_DPA13

DTAC

Does the DPIA provide details of the developer using any new technologies?

Multiple Options: Yes, No, Not Applicable

 ORC_ERC_EDC_DPA14

DTAC

Does the DPIA describe any novel types of processing that the developer is using?

Multiple Options: Yes, No, Not Applicable

 ORC_ERC_EDC_DPA15

DTAC

Does the DPIA describe which screening criteria they have identified/flagged as high risk?

Yes/No

 ORC_ERC_EDC_DPA16

DTAC

Does the DPIA describe the nature of the personal data being processed?

Yes/No

 ORC_ERC_EDC_DPA17

DTAC

Does the DPIA describe the volumes and variety of personal data that is being processed?

Yes/No

 ORC_ERC_EDC_DPA18

DTAC

Does the DPIA describe the sensitive nature of any items of personal data being processed?

Yes/No

 ORC_ERC_EDC_DPA19

DTAC

Does the DPIA provide details of the extent and frequency of the processing?

Yes/No

 ORC_ERC_EDC_DPA20

DTAC

Does the DPIA provide details on the duration of any processing activities covered by the DPIA?

Yes/No

 ORC_ERC_EDC_DPA21

DTAC

Does the DPIA detail the number of data subjects involved in the processing activities described?

Yes/No

 ORC_ERC_EDC_DPA22

DTAC

Does the DPIA provide details of the geographical area covered by the processing activities described?

Yes/No

 ORC_ERC_EDC_DPA23

DTAC

Does the DPIA identify all sources of the data being collected?

Yes/No

 ORC_ERC_EDC_DPA24

DTAC

Does the DPIA describe the nature of the developer’s relationship with the individuals whose data is being processed?

Yes/No

 ORC_ERC_EDC_DPA25

DTAC

Does the DPIA describe how far individuals have control over their data?

Yes/No

 ORC_ERC_EDC_DPA26

DTAC

Does the DPIA describe how far individuals are likely to expect the processing of their data to occur?

Yes/No

 ORC_ERC_EDC_DPA27

DTAC

Does the DPIA provide details on whether the data subjects concerned include children or other vulnerable individuals?

Yes/No

 ORC_ERC_EDC_DPA28

DTAC

Does the DPIA identify any previous experience that the developer has in dealing with the intended type of processing?

Yes/No

 ORC_ERC_EDC_DPA29

DTAC

Does the DPIA identify any relevant advances in the technology and/or security being used in the processing?

Multiple Options: Yes, No, Not Applicable

 ORC_ERC_EDC_DPA30

DTAC

Does the DPIA identify any current issues of public concern with regards to the intended processing?

Yes/No

 ORC_ERC_EDC_DPA31

DTAC

Does the DPIA detail compliances with any UK GDPR codes of conduct or UK certification schemes?

Yes/No

 ORC_ERC_EDC_DPA32

DTAC

Does the DPIA detail whether the developer has considered and complied with relevant codes of practice?

Yes/No

 ORC_ERC_EDC_DPA33

DTAC

Does the DPIA provide details of the developer’s legitimate interests, with regards to the purpose of processing data?

Yes/No

 ORC_ERC_EDC_DPA34

DTAC

Does the DPIA detail the legal basis/bases upon which the developer relies for processing the data collected?

Yes/No

 ORC_ERC_EDC_DPA35

DTAC

Does the DPIA provide details of the intended outcomes of the data processing for the individuals concerned?

Yes/No

 ORC_ERC_EDC_DPA36

DTAC

Does the DPIA provide details of the anticipated benefits of the processing for the developer or for society as a whole?

Yes/No

 ORC_ERC_EDC_DPA37

DTAC

Has the developer sought and documented the views of the individuals whose data will be processed, or their representatives?

Yes/No

 ORC_ERC_EDC_DPA38

DTAC

Does the DPIA provide a justifiable reason for not carrying out consultation with individuals or their representatives?

Yes/No

 ORC_ERC_EDC_DPA39

DTAC

Does the DPIA provide details of consultation with any third party data processors who will be involved in the processing of user data?

Multiple Options: Yes, No, Not Applicable

 ORC_ERC_EDC_DPA40

DTAC

Does the DPIA provide details of consultation with all relevant internal stakeholders?

Yes/No

 ORC_ERC_EDC_DPA41

DTAC

Does the DPIA provide details of any advice being sought from other independent experts?

Yes/No

 ORC_ERC_EDC_DPA42

DTAC

Does the DPIA include details on how the developers plans will help achieve their purpose?

Yes/No

 ORC_ERC_EDC_DPA43

DTAC

Does the DPIA provide evidence that the developer has considered whether there are any other reasonable ways to achieve the same result?

Yes/No

 ORC_ERC_EDC_DPA44

DTAC

Does the DPIA include details on how the developer will prevent function creep?

Yes/No

 ORC_ERC_EDC_DPA45

DTAC

Does the DPIA include details on how the developer will ensure data quality?

Yes/No

 ORC_ERC_EDC_DPA46

DTAC

Does the DPIA include details on how the developer will ensure data minimisation?

Yes/No

 ORC_ERC_EDC_DPA47

DTAC

Does the DPIA include details on how the developer intends to provide privacy information to individuals?

Yes/No

 ORC_ERC_EDC_DPA48

DTAC

Does the DPIA include details on how the developer intends to implement and support individuals’ rights?

Yes/No

 ORC_ERC_EDC_DPA49

DTAC

Does the DPIA include details on the measures that will be taken to ensure any processors comply with their obligations?

Multiple Options: Yes, No, Not Applicable

 ORC_ERC_EDC_DPA50

DTAC

Does the DPIA include details of the safeguarding measures that have been put in place for international transfers of data?

Multiple Options: Yes, No, Not Applicable

 ORC_ERC_EDC_DPA51

DTAC

Has the developer detailed, in the DPIA, all the risks that they have identified relating to each of their processing activities and those relating to third party processors?

Yes/No

 ORC_ERC_EDC_DPA52

DTAC

Are there any additional risks, that you have been able to identify, which have not been included in the DPIA? (please detail)

Multiple Choice

 ORC_ERC_EDC_DPA53

DTAC

Does the DPIA include an assessment of potential security risks?

Yes/No

 ORC_ERC_EDC_DPA54

DTAC

Does the assessment of security risks include the sources of the risks and the potential impacts of the each type of breach?

Yes/No

 ORC_ERC_EDC_DPA55

DTAC

Has an objective approach to assessing the risks been taken through the use of a structured risk matrix that takes into account both the likelihood of harm and the severity of the impact?

Yes/No

 ORC_ERC_EDC_DPA56

DTAC

Has the developer used a different, but acceptable structured approach to objectively assessing the risks associated with their processing activities?

Yes/No

 ORC_ERC_EDC_DPA57

DTAC

Has the developer also considered their own corporate risk, such as the impacts of regulatory action, reputational damage of loss of public trust?

Yes/No

 ORC_ERC_EDC_DPA58

DTAC

Has the developer detailed the source of each risk that they have identified?

Yes/No

 ORC_ERC_EDC_DPA59

DTAC

Has the developer identified the measures that they will put in place in order to mitigate each of the identified risks?

Yes/No

 ORC_ERC_EDC_DPA60

DTAC

Where mitigations have not been put in place, has the developer provided justified reasons for not doing so?

Yes/No

 ORC_ERC_EDC_DPA61

DTAC

Are there any additional mitigations, that you have been able to identify as achievable, that the developer does not appear to have considered? (please detail).

Multiple Options

 ORC_ERC_EDC_DPA62

DTAC

Does the DPIA contain details of the additional measures that the developer planned on taking?

Yes/No

 ORC_ERC_EDC_DPA63

DTAC

Does the DPIA contain a record on whether each of the risks have been removed, reduced, or accepted?

Yes/No

 ORC_ERC_EDC_DPA64

DTAC

Does the DPIA provide details of the overall residual risk, after taking additional measures?

Yes/No

 ORC_ERC_EDC_DPA65

DTAC

Have any of the residual risks been assessed as being “high risk”?

Yes/No

 ORC_ERC_EDC_DPA66

DTAC

Does the DPIA provide details that the need to consult the ICO has been considered?

Yes/No

 ORC_ERC_EDC_DPA67

DTAC

Does the DPIA provide details or evidence of the advice of the DPO being sought, as part of the sign-off process?

Yes/No

 ORC_ERC_EDC_DPA68

DTAC

If the developer has decided not to follow the advice provided by the DPO, have they recorded their reasons?

Multiple Options: Yes, No, Not Applicable

 ORC_ERC_EDC_DPA69

DTAC

Does the app process personal/sensitive Social Care data?

Yes/No

ORC_ERC_EDC_NHS01

ERC

Does the developer collect usage or bug report data?

Yes/No

ORC_ERC_EDC_UBRO01

ERC

Is this collected through informed consent?

Yes/No

ORC_ERC_EDC_UBRO02

ERC

Is this data fully anonymised?

Yes/No

ORC_ERC_EDC_UBRO03

ERC

Does the organisation collect data, through the app, using cookies, web beacons or other similar technologies?

Yes/No

ORC_ERC_EDC_CK01

ERC

Is there a cookie policy?

Yes/No

ORC_ERC_EDC_CK02

ERC

Is there a cookie policy provided separate from the terms and condition and privacy policy?

Yes/No

ORC_ERC_EDC_CK03

ERC

Are users made aware of the use of strictly necessary cookies?

Yes/No

ORC_ERC_EDC_CK04

ERC

Is user consent obtained for the use of non-strictly necessary cookies?

Yes/No

ORC_ERC_EDC_CK05

ERC

Does the organisation keep a log of user consent? (eg. evidence of when consent was obtained and the information provided at the time of consent)

Yes/No

ORC_ERC_EDC_CK06

ERC

Are users informed of how they can easily opt out of the use of cookies?

Yes/No

ORC_ERC_EDC_CK07

ERC

Is the product aimed at children or likely to be used by children?

Yes/No

ORC_ERC_EDC_COP01

ERC

Where consent was the legal basis for processing data was consent, at the time the an individual was a child, then requests for the erasure of data are complied with, whenever possible?

Yes/No

ORC_ERC_EDC_COP04

ERC

Have children been consulted when designing this processing practice?

Yes/No

ORC_ERC_EDC_COP05

ERC

Has the privacy policy been written in plain, age appropriate language?

Yes/No

ORC_ERC_EDC_COP06

ERC

Is consent sought from a responsible parent/guardian?

Yes/No

ORC_ERC_EDC_COP07

ERC

Does the developer ensure they do not seek parental/guardian consent when providing online preventive or counselling services to children?

Yes/No

ORC_ERC_EDC_COP08

ERC

Are there two separate versions of privacy policies, one aimed at the child and the other at the responsible parent/guardian?

Yes/No

ORC_ERC_EDC_COP09

ERC

When marketing the product outside of their country of residence, has the developer taken into consideration other jurisdictional laws regarding children’s privacy (eg. age restrictions)?

Yes/No

ORC_ERC_EDC_COP10

ERC

Has the DPIA been completed with specific details on the assessed risks to children and the mitigations in place?

Yes/No

ORC_ERC_EDC_COP03

ERC

Has a process been designed and put in place that allows children to easily access, understand and exercise their own data protection rights?

Yes/No

ORC_ERC_EDC_COP02

ERC

Is data shared with and processed by any third parties?

Yes/No

ORC_ERC_EDC_PC01

ERC

What services do the third parties provide?

Multiple Choice

ORC_ERC_EDC_PC02

ERC

Is any of the shared data personally identifiable or sensitive?

Yes/No

ORC_ERC_EDC_PC03

ERC

Are there written binding contract agreements between the organisation (controller) and each third party (processor)?

Yes/No

ORC_ERC_EDC_PC04

ERC

Has the developer provided a copy/draft/template of these written agreements?

Yes/No

ORC_ERC_EDC_PC05

ERC

Does the contract clearly define the subject matter and duration of the processing?

Yes/No

ORC_ERC_EDC_PC06

ERC

Does the contract clearly define the purpose of the processing?
Does the contract clearly set out the categories of data subject and the types of data that will be processed?

Yes/No

ORC_ERC_EDC_PC07

ERC

Does the contract clearly set out the categories of data subject and the types of data that will be processed?

Yes/No

ORC_ERC_EDC_PC14

ERC

Does the contract clearly describe the obligations and rights of the controller?

Yes/No

ORC_ERC_EDC_PC08

ERC

Does the contract make clear that the processor must only act upon written instruction from the controller?

Yes/No

ORC_ERC_EDC_PC09

ERC

Does the contract make clear to the processor their responsibilities in ensuring that their employees are subject to a duty of confidence?

Yes/No

ORC_ERC_EDC_PC10

ERC

Does the contract make clear to the processor their responsibilities in ensuring all appropriate safeguarding measures are in place to ensure the security of the data they are processing?

Yes/No

ORC_ERC_EDC_PC11

ERC

Does the contract clearly state that the processor must assist the controller when responding to requests to exercise user rights?

Yes/No

ORC_ERC_EDC_PC12

ERC

Does the contract clearly state that the processor must assist the controller in meeting their legal data protection requirements? (eg. notifying of breaches and completing DPIAs).

Yes/No

ORC_ERC_EDC_PC13

ERC

CLINICAL EVIDENCE

Evidence of Effectiveness

The Evidence Standards for Digital Health Technologies Framework (“ESF”) was created by the UK’s National Institute for Health and Care Excellence (“NICE”). This framework clustered DHT’s into relevant Tiers and identified for each Tier what forms of ‘evidence’ or ‘assurance’ would be required. It is therefore better to think of the ESF as an Assurance Standards Framework, with evidence being just one of many elements within that digital assurance matrix.

An adapted version of the ESF has been developed over time with and has now been adopted in numerous other national and pan-national Digital Health Assessment Frameworks in areas like New Zealand, Canada, Israel and the Netherlands. We conduct an analysis of any evidence available through the Review Resources. If this exists, the app is evaluated against a series of questions to determine the quality of this evidence. We look for:

·         a suitable sample size and make up;

·         a p value of below 0.05 to indicate significance;

·         a p value below 0.02 for near significance; and

·         an appropriate comparator.

This is scaled against the NICE Evidence Standards Framework and we look for a higher level of evidence for apps with more complex functionality and higher risk.

Question

Answer Type

ORCHA Question Reference Source

Framework

EE02: What type/s of evidence is available?

Survey, RCT, Pilot study, Observational (Case study, Cross-sectional, Cohort), Meta-Analysis/Systematic Review

Multiple Choice

ORC_EE02

OBR

How many pieces of evidence does the app provide?

Multiple Option

ORC_EE14

OBR

How many RCT's and/or observational studies does the app have?

Multiple Option

ORC_EE13

OBR

For each type of relevant evidence:

EE10: What category does the evidence relate to?

Free Text

ORC_EE10 

OBR

For each type of relevant evidence:

EE11: What benefit does the evidence relate to?

Free Text

ORC_EE11          

OBR

EE03: Provide links to the publicly available evidence/published evidence that the developer has provided.

Free Text

ORC_EE03

OBR

For each type of relevant evidence:

EE04: Is the sample size appropriate?

Yes/No

ORC_EE04          

OBR

For each type of relevant evidence:

EE05: Does the evidence found provide a p-value?

Yes/No

ORC_EE05

OBR

For each type of relevant evidence:

EE06: Does the P-Value demonstrate significance (p<0.05)?

Yes/No

ORC_EE06

OBR

For each type of relevant evidence:

EE12: Does the P-Value demonstrate near significance (p<0.2)?

Yes/No

ORC_EE12

OBR

For each type of relevant evidence:

EE07: Is there a comparator?

Yes/No

ORC_EE07

OBR

For each type of relevant evidence:

EE08: Is the comparator validated?

Yes/No

ORC_EE08

OBR

Behavioural Change

There are some scenarios where the app utilises widely accepted techniques with a breadth of evidence. In this instance the app may not deem it appropriate to fund a full randomised control trial to demonstrate effectiveness. Therefore we give some value points for fully referencing evidence for behavioural change techniques used within the app. This is not however treated in the same way as where the app has provided direct evidence of its own effectiveness.

Question

Answer Type

ORCHA Question Reference Source

Framework

Does the App have its own high quality study?

Yes/No

ORC_BCT01  

OBR

Does the App reference and evidence its behaviour change technique?

Yes/No

ORC_BCT02

OBR

Professional Backing

We look for evidence of an appropriate professional being involved in the app's design and development, or if the app has been externally accredited. A relevant professional is deduced in the context of an app. For example, for a simple yoga app we would accept a qualified yoga instructor as a relevant professional, but for a complex clinical solution we would only accept a relevantly qualified clinician. External accreditations are wide ranging, but we would look for an appropriate body, for example the British Heart Foundation giving an endorsement to a cardiology app.

Question

Answer Type

ORCHA Question Reference Source

Framework

Is there a suitably qualified Professional involved in the Development team of the App?

Yes/No

ORC_PB01

OBR

Does the organisation behind the App have relevant credentials? 

Yes/No

ORC_PB02

OBR

Is there evidence of an endorsement by a relevant body?

Yes/No

ORC_PB03

OBR

Are organisations using the App?

Yes/No

ORC_PB04

OBR

Is there a statement that it has been positively evaluated or validated by a relevant healthcare professional?

Yes/No

ORC_PB05

OBR

Please specify who the relevant experts are and what qualifications they hold.   

Free Text

ORC_AE17·          

OBR

Is there evidence within the app that the developer has validated any guidance with relevant reliable information sources or references? 

Yes/No

ORC_PB06

OBR

ORCHA Adapted ESF Compliance

The first part of this section assesses which ESF Tier the app falls under, and is non-scoring. The second part assesses whether the app meets the minimum requirements of that Tier. Compliance with the ESF is determined by the app answering positively to all questions that have been flagged as a requirement for its Tier of the ESF and all Tiers below.

 

Question

Answer Type

ORCHA Question Reference Source

Framework

What Tier of the ESF is the App?

 

 

OBR

Is the app Tier 1?

Yes/No

ORC_ESF01

OBR

Is the app Tier 2a?

Yes/No

ORC_ESF02

OBR

Is the app Tier 2b?

Yes/No

ORC_ESF03

OBR

Is the app Tier 3a?

Yes/No

ORC_ESF04

OBR

Is the app Tier 3b?

Yes/No

ORC_ESF05

OBR

Has the App met:

 

 

OBR

Tier 1 minimum requirements?

Yes/No

ORC_ESF06

OBR

Tier 2a requirements?

Yes/No

ORC_ESF07

OBR

Tier 2b requirements?

Yes/No

ORC_ESF08

OBR

Tier 3a requirements?

Yes/No

ORC_ESF09

OBR

Tier 3b requirements?

Yes/No

ORC_ESF10

OBR

Does the app have appropriate evidence for the ESF tier?

Yes/No

ORC_ESF11

OBR

Medical Devices

It is proposed that the NORDIC Assessment assess if the app is likely to be a medical device under the current guidance from the MDR (https://ec.europa.eu/growth/sectors/medical-devices_en ). We then evaluate if the app displays the relevant CE mark.

Question

Answer Type

ORCHA Question Reference Source

Framework

Is the app a medical device?

Yes/No

ORC_MD11

OBR

Does the app have a CE mark?    

Yes/No

ORC_AE06

OBR

Does the app state that it has been assessed by the MHRA or other relevant body, and does not require a CE mark? 

Yes/No

ORC_AE08

OBR

What class is the app certified as?           

Free Text

ORC_MD09

OBR

Has the app been FDA approved? (Food and Drug Administration)  

Yes/No

ORC_FDA01

OBR

Has the app been FDA cleared?             

Yes/No

ORC_FDA02

OBR

Clinical Safety/Risk Management

It is proposed that the evaluation looks for any safeguarding measures in communication functions of the app, if relevant.

Question

Answer Type

ORCHA Question Reference Source

Framework

Is there a statement or any evidence showing that appropriate safeguarding measures are in place around peer-support and other communication functions within the platform?

(Tier 2a requirement

Only asked of apps that require such measures because of it functional capabilities / intended purpose )

Yes/No

ORC_AE13

OBR

Does the Developer clearly identify who the app should and should not be used by? 

Yes/No

ORC_S01

OBR

Does the Developer publish their risk management processes?

Yes/No

ORC_S02

OBR

Does the Developer make clear risks associated with using the app?

Yes/No

ORC_S03

OBR

Is there a way for the user to confirm that the data input is accurate?

Yes/No

ORC_S04

OBR

Does the Developer list a Clinical Safety Officer on/in any relevant sites/content?

Yes/No

ORC_S05

OBR

Please provide more detail.

Free Text

ORC_S06

OBR

Is the App/Solution in scope for a clinical safety assessment?

Yes/No

ORC_ERC_OCSA_CSS1

OCSA

Why is the app in scope?

Free Text

ORC_ERC_OCSA_CSS2

OCSA

Has the Developer provided a thorough summary about why the App is out of scope? (Eg is it complete? Does it match the functionality put forward?)

Yes/No

ORC_ERC_OCSA_CSS3

OCSA

What risks if any, have been documented concerning harm to a patient?

Free Text

ORC_ERC_OCSA_CSS4

OCSA

If the App/Solution has been deemed in scope, has the Developer supplied suitable Risk Management Documentation?

Yes/No

ORC_ERC_OCSA_CSS5

OCSA

Please confirm the name of your Clinical Safety Officer (CSO), their profession and registration details?

Free Text

ORC_DTAC_CS120

OCSA

Is the CSO a suitably qualified and experienced clinician?

Yes/No

ORC_ERC_OCSA_CSO1

OCSA

Does the named CSO have appropriate qualifications and up to date registration details?

Yes/No

ORC_DTAC_CS121

OCSA

Is there evidence the CSO has played an active part in the clinical safety process – approval of the risk management file, hazard assessment participation etc?

Yes/No

ORC_ERC_OCSA_CSO3

OCSA

Has the developer provided their Clinical Safety Case and Hazard Log?

Yes/No

ORC_DTAC_CS112

OCSA

Does the Developer outline the need for the Risk Management Documentation?

Yes/No

ORC_ERC_OCSA_CSD5

OCSA

Does it have full version history and issue date published?

Yes/No

ORC_ERC_OCSA_CSD2

OCSA

Has the Developer described their clinical risk management system? ( identification of key personnel, their roles and responsibilities; identification of clinical risk management governance structure.

Yes/No

ORC_ERC_OCSA_CSD6

OCSA

Does the Safety Case make a mention of a test summary? (Summary of any outstanding test issues and the impact on clinical safety)

Yes/No

ORC_ERC_OCSA_CSD7

OCSA

Does the Hazard Log have full version history and issue date published?

Yes/No

ORC_ERC_OCSA_CSD8

OCSA

Is there evidence of a CSO reviewing, contributing or approving the Safety Case?

Yes/No

ORC_ERC_OCSA_CSD3

OCSA

Does the Hazard Log have full version history and issue date published?

Yes/No

ORC_ERC_OCSA_CSD9

OCSA

Are the hazards listed complete? (From a review of the listed hazards, do they have all of the details required completed such as name, clinical impact and risk ratings. Do the risk ratings look appropriate or do they appear to be copy and paste throughout the listed hazards? Are they scored on the low side? Does the consequence change pre and post assessment?

Yes/No

ORC_ERC_OCSA_CSD4

OCSA

Are the potential harms related to the user/patient?

Yes/No

ORC_ERC_OCSA_CSD11

OCSA

Do the harms outline what the clinical impact may be for the user?

Yes/No

ORC_ERC_OCSA_CSD12

OCSA

Has the Developer covered all of the possible causes for each Hazard?

Yes/No

ORC_ERC_OCSA_CSD13

OCSA

Do the risk ratings look appropriate or do they appear to be copy and paste throughout the listed hazards?

Yes/No

ORC_ERC_OCSA_CSD14

OCSA

Are hazards split incorrectly into potential and actual harm?

Yes/No

ORC_ERC_OCSA_CSD15

OCSA

Is there evidence of a CSO reviewing, contributing or approving the hazard log?

Yes/No

ORC_ERC_OCSA_CSD10

OCSA

Has the Developer implemented the clinical risk analysis activities defined in the Clinical Risk Management Plan?

Yes/No

ORC_ERC_OCSA_CRAP1

OCSA

Is the Clinical Risk Analysis carried out by a multi-disciplinary group?

Yes/No

ORC_ERC_OCSA_CRAP2

OCSA

Has the Developer defined the clinical scope of the Health IT System which is to be delivered?

Yes/No

ORC_ERC_OCSA_CRAP3

OCSA

Has the Developer defined the intended use of the Health IT System which is to be delivered?

Yes/No

ORC_ERC_OCSA_CRAP4

OCSA

Does the Risk Management Documentation make it clear where the App fits into the Clinical Workflow – How would a patient use the app appropriately to become well again?

Yes/No

ORC_ERC_OCSA_CRAP5

OCSA

Has the Developer outlined Third Party Products integrated within the Health IT System to be released?

Multiple Options: Yes, No, Not Applicable

ORC_ERC_OCSA_DSP1

OCSA

Has the Developer deploying the Health IT System considered how it will impact on the current business processes and ways of working?

Yes/No

ORC_ERC_OCSA_DSP2

OCSA

Is there usability and human factors related evidence within the scope?

Yes/No

ORC_ERC_OCSA_DSP3

OCSA

Has the Developer assessed any infrastructure at the Health Organisation that is within their scope of influence, required to support the deployment of the Health IT System? (This may be achieved by the Manufacturer specifying the minimum system requirements)

Multiple Options: Yes, No, Not Applicable

ORC_ERC_OCSA_DSP4

OCSA

Where data migration is to be undertaken by the Developer it should be included in the scope of the clinical risk management activities. Is the Developer undertaking any Data Migration?

Yes/No

ORC_ERC_OCSA_DSP5

OCSA

Is the Data Migration being undertaken by the Developer properly covered in the documentation?

Yes/No

ORC_ERC_OCSA_DSP6

OCSA

Has the Developer identified any hazards associated with the data migration, analysed and suitably mitigated? (working in conjunction with the relevant Health Organisation as appropriate)

Yes/No

ORC_ERC_OCSA_IPH1

OCSA

Has the Developer considered the end to end clinical process, including functionality and how that functionality is used?

Yes/No

ORC_ERC_OCSA_IPH2

OCSA

Has the Developer considered inter and intra Health IT System messaging?

Yes/No

ORC_ERC_OCSA_IPH3

OCSA

Has the Developer assessed the health IT system architecture and design?

Yes/No

ORC_ERC_OCSA_IPH4

OCSA

Is there a clear matrix, which is used to define the risk ratings?

Yes/No

ORC_ERC_OCSA_CRE11

OCSA

For each identified hazard, has the Developer evaluated whether the initial clinical risk is acceptable?

Yes/No

ORC_ERC_OCSA_CRE9

OCSA

Has the Developer used the risk acceptability criteria previously defined?

Yes/No

ORC_ERC_OCSA_CRE10

OCSA

Has the Developer identified appropriate clinical risk control measures to remove any unacceptable clinical risk?

Yes/No

ORC_ERC_OCSA_CRE1

OCSA

Has the Developer assessed Proposed clinical risk control measures to determine whether new hazards will be introduced as a result of the measures?

Yes/No

ORC_ERC_OCSA_CRE2

OCSA

Has the Developer assessed proposed clinical risk control measures to determine whether the clinical risks for previously identified hazards will be affected?

Yes/No

ORC_ERC_OCSA_CRE3

OCSA

Is the Developer managing new hazards, or increased clinical risks?

Multiple Options: Yes, No, Not Applicable

ORC_ERC_OCSA_CRE4

OCSA

For each identified hazard, has the Developer MUST evaluated whether the residual clinical risk is acceptable?

Yes/No

ORC_ERC_OCSA_CRE5

OCSA

Has the Developer used the risk acceptability criteria previously defined?

Yes/No

ORC_ERC_OCSA_CRE6

OCSA

If the residual clinical risk is unacceptable, has the Developer identified additional clinical risk control measures in order to reduce the clinical risk?

Multiple Options: Yes, No, Not Applicable

ORC_ERC_OCSA_CRE7

OCSA

If the Developer has determined that no suitable risk control measures are possible, have they conducted a clinical risk benefit analysis of the clinical risk?

Multiple Options: Yes, No, Not Applicable

ORC_ERC_OCSA_CRE8

OCSA

Has the Developer’s analysis shown that the clinical benefits of the intended use outweigh the residual clinical risk?

Yes/No

ORC_ERC_OCSA_CRB1

OCSA

Has the Developer implemented the clinical risk control measures identified? (except where these are to be implemented by another organisation.)

Yes/No

ORC_ERC_OCSA_CRCM1

OCSA

Have the clinical risks from all identified hazards been considered and accepted?

Yes/No

ORC_ERC_OCSA_CRCM2

OCSA

Have any hazard rating reductions been fully justified?

Yes/No

ORC_ERC_OCSA_CRCM3

OCSA

USABILITY & ACCESSIBILITY

Design and Development

This considers the design and development of the app and whether it follows any recognised app design standards, such as WC3, WCAG 2.0 AA, WCAG 2.1 AA, ISO 9241, Apple HIG, or Android App Quality Guidelines. The NordDEC also considers whether there was any user involvement during the development of the app, user involvement in testing, or if any features were based on user feedback.

Question

Answer type

ORCHA Question Reference Source

Framework

Is there a statement within the app outlining compliance with any currently recognised app design standards?      WC3

·         WCAG 2.0 AA

·         WCAG 2.1 AA

·         ISO 9241

·         Apple HIG

·         Android App Quality Guidelines

Multiple Choice

ORC_DE01

OBR

Is there a statement about user feedback during design/development?

Yes/No

ORC_DT01

OBR

Is there any evidence of user involvement in testing?   

Yes/No

ORC_DT02

OBR

(Web-Apps Only)
Has the App been designed to work on Mobile Devices and Tablets?

Yes/No

ORC_U01a

OBR

Accessibility

Accessibility is important to consider, as the app should be accessible to all users regardless of their specific needs. The NordDEC considers whether the app is customisable to suit certain needs, such as poor sight or hearing impairments. If the app uses any specialist or medical terms, these should be clearly explained to the user.

Question

Answer type

ORCHA Question Reference Source

Framework

Can the user change the font size in-app/does the app respond to device preferences?    

Yes/No

ORC_U04

OBR

Does the app provide support for users with poor sight?     

Yes/No

ORC_U07

OBR

Does the app provide support for users with hearing difficulty?        

Yes/No

ORC_U08

OBR

(Web-Apps Only)
Does the App provide users with an Accessibility Statement?

Yes/No

ORC_U26

OBR

Usability

This also ties in to the accessibility of the app, including further customisation options. The NordDEC identifies if the app has any functions to aid navigation, such as a home button, back button, help button or search feature. If the app utilises push or email notifications, the NordDEC identifies whether the user has options to manage these for their own preference or privacy, both at app level and at device level. Finally, if there are any bugs identified during evaluation, this will be flagged. If the app contains a forum, then we look for a statement to ensure that forum content is moderated.

Question

Answer type

ORCHA Question Reference Source

Framework

Can the user change the presentation theme?

Yes/No

ORC_U06

OBR

Does the app include the following functions:

·         Home/Menu button

·         Back button

·         Help/About button

·         Search button

Multiple Choice

ORC_U32

OBR

Are any medical, specialist or technical terms explained clearly to the user?

Yes/No

ORC_U15

OBR

Does the app send push notifications?

Yes/No

ORC_D29

OBR

Does the app send email notifications?

Yes/No

ORC_D30

OBR

Does the user have options to manage the notification settings (push/email) within the app for convenience/privacy? 

Yes/No

ORC_D31

OBR

Does the app inform the user how to manage notification settings for convenience/privacy (to prevent info being shown if device is locked but on show)? (android only question)

Yes/No

ORC_D32

OBR

Was there any evidence of bugs during evaluation?

Yes/No

ORC_U23

OBR

Support

Support is a key area of this section, as it is important that users are informed of ways in which they can contact the developer should they have any problems or questions with the app. The NordDEC also identifies what type of support is offered to users, and if there is a commitment from the developer to respond to any user queries. We would expect to see that the type of support offered is appropriate to the app level - a higher level app would therefore require a more sophisticated offer of user support.

Question

Answer type

ORCHA Question Reference Source

Framework

If there is a forum, is there a statement within the app that the forum content is moderated?

Yes/No

ORC_FC03

OBR

Is there a statement about how to report issues to the developer?

Yes/No

ORC_U24

OBR

What kind of support is offered?

Multiple Choice

ORC_U33

OBR

Is there any statement within the app about the developer’s commitment to addressing problems reported to them? (e.g. timescales to respond, commitment to eradicate reported bugs and faults)       

Yes/No

ORC_U25

OBR

SECURITY & TECHNICAL STABILITY

Technical Stability

Question

Answer Type

ORCHA Question Reference Source

Framework

Does the App connect to an internet-based API (e.g. App Developer Web Service, Social Media, Adverts)?

Yes/No

ORC_ERC_OTS_C01

 ERC

List the APIs

Free Text

ORC_ERC_OTS_C02

ERC

Does the App connect to a medical device?

Yes/No

ORC_ERC_OTS_C03

ERC

Does the App connect to NHS Services (e.g. SPINE)?

Yes/No

ORC_ERC_OTS_C04

ERC

Does the App operate without wi-fi

Yes/No

ORC_ERC_OTS_C05

ERC

Does the App operate without cellular network

Yes/No

ORC_ERC_OTS_C06

ERC

Is the platform Web based or Mobile?

Multiple Option

ORC_ERC_OTS_D04

ERC

Does the App access, process or store Personal and/or Sensitive Data?

Yes/No

ORC_ERC_OTS_D01

ERC

Is sensitive data persisted to the mobile device?

Yes/No

ORC_ERC_OTS_D02

ERC

Does the App access, process or store Personal and/or Sensitive Data?

Multiple Choice

ORC_ERC_OTS_D03

ERC

What Permissions does the App request

Free Text

ORC_ERC_OTS_P01

ERC

Does the App provide Alerts or Notifications?

Yes/No

ORC_ERC_OTS_OTF01

ERC

Does the App provide Suggestions?

Yes/No

ORC_ERC_OTS_OTF02

ERC

Does the App undertake calculations?

Yes/No

ORC_ERC_OTS_OTF03

ERC

Are the source code and any configuration items for the product version controlled with all changes audited?

Yes/No

ORC_ERC_OTS_PSL01

ERC

Provide details of any associated processes / procedures and tools that are used.

Free Text

ORC_ERC_OTS_PSL02

ERC

Do you have the capacity to rollback to previous versions of your product?

Yes/No

ORC_ERC_OTS_PSL03

ERC

Provide details of any associated processes / procedures and tools that are used.

Free Text

ORC_ERC_OTS_PSL04

ERC

Are the processes for accepting and responding to technical faults from end users appropriate?

Yes/No

ORC_ERC_OTS_PSL05

ERC

Do you provide on-line support for user queries

Free Text

ORC_ERC_OTS_PSL06

ERC

Do you proactively monitor running of systems and system components to automatically identify faults and technical issues?

Yes/No

ORC_ERC_OTS_PSL07

ERC

Provide details of any associated processes / procedures and tools that are used.

Free Text

ORC_ERC_OTS_PSL08

ERC

Do you have a documented roadmap for future development of your product?

Yes/No

ORC_ERC_OTS_PSL09

ERC

Provide details of planned development, technical updates.

Free Text

ORC_ERC_OTS_PSL10

ERC

Does the Developer provide details of how they will ensure the continued availability of their product?

Yes/No

ORC_ERC_OTS_PSL11

ERC

Do you have a plan for decommissioning your product?

Yes/No

ORC_ERC_OTS_PSL12

ERC

Describe your processes for decommissioning your product and dealing with any identifiable data.

Free Text

ORC_ERC_OTS_PSL13

ERC

Do you have a plan for dealing with any identifiable data in the event that an individual stops using your product? For example by installing or unsubscribing.

Yes/No

ORC_ERC_OTS_PSL14

ERC

Provide details of any associated processes / procedures and tools that are used.

Free Text

ORC_ERC_OTS_PSL15

ERC

Does the organisation follow any formal testing standards

Yes/No

ORC_ERC_OTS_PSL16

ERC

Provide details of any associated processes / procedures and tools that are used.

Free Text

ORC_ERC_OTS_PSL17

ERC

For each of the following if they are carried out please describe the people / roles that are involved, the processes that they work to even if they are informal.

-

ORC_ERC_OTS_PSL18

ERC

Unit

Free Text

ORC_ERC_OTS_PSL19

ERC

Regression

Free Text

ORC_ERC_OTS_PSL20

ERC

 End-to-end / Integration

Free Text

ORC_ERC_OTS_PSL21

ERC

User Acceptance

Free Text

ORC_ERC_OTS_PSL22

ERC

A/B

Free Text

ORC_ERC_OTS_PSL23

ERC

PEN / Vulnerability

Free Text

ORC_ERC_OTS_PSL24

ERC

Testing across devices

Free Text

ORC_ERC_OTS_PSL25

ERC

Load / Performance

Free Text

ORC_ERC_OTS_PSL26

ERC

Security

Free Text

ORC_ERC_OTS_PSL27

ERC

Other non-functional tests

Free Text

ORC_ERC_OTS_PSL28

ERC

Other testing

Free Text

ORC_ERC_OTS_PSL29

ERC

Has the Developer provided sufficient evidence to satisfy all the requirements of the product's testing?

Yes/No

ORC_ERC_OTS_PSL30

ERC

 

Technical Security

Question

Answer Type

ORCHA Question Reference Source

Framework

Does the organisation have ISO27001:2013 accreditation?

Yes/No

ORC_ERC_SEC_ORG1

ERC

Is the certification body in the UKAS list of ISO27001:2013 certification bodies?

Yes/No

ORC_ERC_SEC_ORG2

ERC

Please provide statement of applicability?

Free Text

ORC_ERC_SEC_ORG3

ERC

Does the scope include product and associated services?

Yes/No

ORC_ERC_SEC_ORG4

ERC

Is the Application a Native Application for a Mobile Device?

Yes/No

ORC_ERC_SEC01

ERC

Is the Application a Web Application?

Yes/No

ORC_ERC_SEC02

ERC

Are Web API’s accessed?

Yes/No

ORC_ERC_SEC03

ERC

Does the App access, process or store Personal and/or Sensitive Data?

Multiple Choice

ORC_ERC_SEC04

ERC

Is sensitive data persisted to the mobile device?

Yes/No

ORC_ERC_SEC05

ERC

What Permissions does the Application request?

Multiple Choice

ORC_ERC_SEC06

ERC

OWASP Level is required in order to review evidence.

IF Mobile = Y

IF 'Personal and /or Sensitive Data is accessed, processed or stored’ = Y

OWASP Level then MASVS = 2

IF Sensitive data is persisted to the device then MASVS = 2+R

ELSE

OWASP Level then MASVS = 1

IF Web = Y

IF 'Personal and /or Sensitive Data is accessed, processed or stored' = Y

OWASP Level then ASVS= 2

ELSE

OWASP Level then ASVS= 1

Multiple Options

ORC_ERC_SEC07
(What OWASP Level is the App)

ERC

Does the Application connect to a Medical device?

Yes/No

ORC_ERC_SEC08

ERC

Does the Application connect to NHS Service?

Yes/No

ORC_ERC_SEC09

ERC

Does the Application provide Alerts or Notifications?

Yes/No

ORC_ERC_SEC10

ERC

Does the Application provide Suggestions?

Yes/No

ORC_ERC_SEC11

ERC

Does the Application undertake Calculations?

Yes/No

ORC_ERC_SEC12

ERC

Does the Application support in-App purchases?

Yes/No

ORC_ERC_SEC13

ERC

Has a Security Assessment been undertaken by an accredited external third-party?

Yes/No

ORC_ERC_SEC14

ERC

Is the external third-party a CREST / APMG / CHECK registered supplier?

Multiple Options

ORC_ERC_SEC15

ERC

Does the scope of the report cover the full Technical Architecture of Application?

Yes/No

ORC_ERC_SEC16

ERC

Has an industry-standard been used for the risk model in the associated PEN/Vulnerability testing?

Yes/No

ORC_ERC_SEC17

ERC

Have all ‘Medium’ Risks / Issues identified been mitigated and resolved; and can this be demonstrated through retesting within six weeks from the original PEN / Vulnerability testing?

Yes/No

ORC_ERC_SEC18

ERC

Has the Code-Level Security Assessment been undertaken against the correct OWASP Level?

Yes/No

ORC_ERC_SEC19

ERC

Is the methodology for the Security Review proportional to the attack service and risk of the Application?

Yes/No

ORC_ERC_SEC20

ERC

Annex 1

Answer Type

Meaning

Yes/No

These are questions where the answer is a binary yes or no.

Free Text

These questions allow us to collect exact information, such as links to the evidence in the journal, or who exactly has provided the testimonial.

Multiple Choice

Multiple choice questions are for where options fall only in a binary category - eg Data Type, Research Type, and more than one can be selected

Multiple Options

Multiple options questions are for where options fall only in a binary category - eg Data Type, Research Type, and only one can be selected

 

 

The project is run by N!P, jointly funded by Nordic Innovation and Nordic healthtech industry and powered by ORCHA.