Skip to main content
0 / 14
Skip to - Close

Quality Rules

Section outline

  • Quality Rules

    In this section we will take a practical look at the Quality Rules part of Stratio Data Governance.

      1. From Stratio Rocket, import the Trusted Data Workflow from the Material section into your working folder within your Rocket project. This workflow reads the practica_paciente and practica_episodio tables from Postgres and writes them to an HDFS folder, simulating a simple process of generating a trusted information area.

      2. Adapt and run the workflow so it writes to a folder called trusted_data within your personal HDFS folder (/certification/governance/students/[USER]/trusted_data/). You only need to modify the Path configuration of the Parquet output element.

      3. Return to Stratio Governance, launch an automatic autodiscovery over your personal working folder so it syncs as soon as possible.

      4. Browse the data dictionary to the folder trusted_data you just created and access the technical lineage of the two tables.


      5. Create a Quality Rule (QR)Proactive (Type "Rows") for completeness of the NHC column in the paciente table in trusted_data following the recommended standard naming convention:

                                    [USERNAME].[PROJECT/CLIENT].[AREA].[TABLE].[COLUMN].[EXECUTE].[TYPE].[CATEGORY].[ACTION].[COUNTER]


      (e.g., eteniente.stratio.TD.practica_paciente.NHC.PR.B.Completeness.PT.001). These are the necessary details for implementing the quality rule:

             - Acceptance threshold by ranges: < 80% = KO

                                                                                                        80% <> 90% = Warning

                                                                                                        >= 90% = OK

             - Action: Pass Through

              - Audit: No


              6. Re-run the WF that generates trusted_data to see how the QR runs and review its results after execution.

              7. Similarly, following the previously indicated naming convention, create a Proactive QR (Type "Rows") for format for the following fields and specifications:
                 - centro in trusted_data>practica_episodio (data pattern: C[0-9][0-9][0-9][0-9]) and email in trusted_data>practica_paciente (data pattern: ^[^@]+@[^@]+\.[a-zA-Z]{2,}$)
                 - Acceptance threshold: = 100%
                 - Action: Pass Through
                 - Audit: No

             8. Re-run the WF that generates trusted_data to see how the 3 QRs are executed and review their results.

             9. Following the previously indicated naming convention, create a QR Scheduled (Type "Table") for permitted range values for the tipo column in the practica_episodio_csv table in the Technical View of your personal collection. These are the necessary details for the quality rule implementation:
                 - Acceptance threshold: > 80%
                 - Target dataset: all table records
                 - Permitted values for the condition: ('URGENCIAS', 'HOSPITALIZACION')
                 - It is important to schedule the QR to run at a specific future date and time (preferably 1 or 2 minutes from now) and with no repetitions.

             10. Review the schedule and execution results in Rocket in the "System" project where scheduled quality rules are orchestrated.

             11. Once executed, reschedule it to run on the 1st and 15th of every month.

             12. The following dashboard was generated in Discovery to monitor the execution of the quality rules: https://discovery.practices.stratio.com/discovery/dashboard/1-dashboard-monitorizacion-de-calidad. You must use the available filters until you only see the quality rules you just created.

             13. Add the Custom Attribute "objetivo_gobierno_[USER]" previously created to the 3 QRs you have created.

             14. Export the QRs you have created to attach them in the submission.

Open in new window