-

 

Types of data validation

This reading describes the purpose, examples, and limitations of six types of data validation. The first five are validation types associated with the data (type, range, constraint, consistency, and structure) and the sixth type focuses on the validation of application code used to accept data from user input. 

As a junior data analyst, you might not perform all of these validations. But you could ask if and how the data was validated before you begin working with a dataset. Data validation helps to ensure the integrity of data. It also gives you confidence that the data you are using is clean. The following list outlines six types of data validation and the purpose of each and includes examples and limitations.

Icon of a question mark with the words data type

  • Purpose: Check that the data matches the data type defined for a field.

  • Example: Data values for school grades 1-12 must be a numeric data type.

  • Limitations: The data value 13 would pass the data type validation but would be an unacceptable value. For this case, data range validation is also needed.

Icon of a man holding a magnifying glass with words data range

  • Purpose: Check that the data falls within an acceptable range of values defined for the field.

  • Example: Data values for school grades should be values between 1 and 12.

  • Limitations: The data value 11.5 would be in the data range and would also pass as a numeric data type. But, it would be unacceptable because there aren't half grades. For this case, data constraint validation is also needed.

Icon with gears and the words data constraints

  • Purpose: Check that the data meets certain conditions or criteria for a field. This includes the type of data entered as well as other attributes of the field, such as number of characters.

  • Example: Content constraint: Data values for school grades 1-12 must be whole numbers.

  • Limitations: The data value 13 is a whole number and would pass the content constraint validation. But, it would be unacceptable since 13 isn’t a recognized school grade. For this case, data range validation is also needed.

Icon of a traffic light with words data consistency

  • Purpose: Check that the data makes sense in the context of other related data.

  • Example: Data values for product shipping dates can’t be earlier than product production dates.

  • Limitations: Data might be consistent but still incorrect or inaccurate. A shipping date could be later than a production date and still be wrong.

Icon of a clipboard checklist with words Data structure

  • Purpose: Check that the data follows or conforms to a set structure.

  • Example: Web pages must follow a prescribed structure to be displayed properly.

  • Limitations: A data structure might be correct with the data still incorrect or inaccurate. Content on a web page could be displayed properly and still contain the wrong information.

Icon with computer arrows traveling to multiple users with the word Code validation

  • Purpose: Check that the application code systematically performs any of the previously mentioned validations during user data input.

  • Example: Common problems discovered during code validation include: more than one data type allowed, data range checking not done, or ending of text strings not well defined.

  • Limitations: Code validation might not validate all possible variations with data input.

Comments

Post a Comment

Popular posts from this blog

Hands-on Activity: Introduction to Kaggle (Google Data Analyst Certificate on Coursera)

-
Image
  1 . Question 1 Activity overview By now, you’ve learned a lot about different data types and data structures. In this activity, you will work with datasets from Kaggle, an online community of people passionate about data. To start this activity, you’ll create a Kaggle account, set up a profile, and explore Kaggle notebooks. Every data analyst has a data community that they rely on for help, support, and inspiration. Kaggle can help you build your own data community.  Kaggle has millions of users in all stages of their data career, from beginners to data scientists with decades of experience. The Kaggle community brings people together to develop their data analysis skills, share datasets and interactive notebooks, and collaborate on solving real-life data problems.  Check out this brief introductory video to learn more about Kaggle.  By the time you complete this activity, you will be able to use many of Kaggle’s key features. This will enable you to create notebo...
Read more

Transforming data

-
Image
  Transforming data What is data transformation? A woman presenting data, a hand holding a medal, two people chatting, a ship's wheel being steered, two people high-fiving each other In this reading, you will explore how data is transformed and the differences between wide and long data. Data transformation is the process of changing the data’s format, structure, or values. As a data analyst, there is a good chance you will need to transform data at some point to make it easier for you to analyze it.  Data transformation usually involves: Adding, copying, or replicating data  Deleting fields or records  Standardizing the names of variables Renaming, moving, or combining columns in a database Joining one set of data with another Saving a file in a different format. For example, saving a spreadsheet as a comma separated values (CSV) file. Why transform data? Goals for data transformation might be:  Data organization : better organized data is easier to use Data c...
Read more