{"id":31963,"date":"2021-01-14T11:43:08","date_gmt":"2021-01-14T11:43:08","guid":{"rendered":"https:\/\/www.testpreptraining.com\/tutorial\/?page_id=31963"},"modified":"2021-01-14T11:43:09","modified_gmt":"2021-01-14T11:43:09","slug":"using-data-flows","status":"publish","type":"page","link":"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/","title":{"rendered":"Using Data Flows"},"content":{"rendered":"\n<p>The process of mapping data flows refers to visually designed data transformations in the Azure Data Factory. Also, the process of data flow permits data engineers to develop data transformation logic without writing code. Thereby, the resulting data flow are executed as activities within Azure Data Factory pipelines that use scaled-out Apache Spark clusters. Some of the important features of data flow include &#8211; <\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Firstly, data flow activities can be operationalized using existing Azure Data Factory scheduling, control, flow, and monitoring capabilities.<\/li><li>Secondly, mapping data flow provides an entirely visual experience with no coding required. <\/li><li>Thirdly, data flow runs on ADF-managed execution clusters for scaled-out data processing. <\/li><li>Lastly, Azure Data Factory handles all the code translation, path optimization, and execution of your data flow jobs.<\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Steps to Create Data Flows<\/strong><\/h4>\n\n\n\n<div class=\"wp-block-image is-style-default\"><figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/media\/data-flow\/new-data-flow.png\" alt=\"New data flow\" width=\"684\" height=\"276\"\/><figcaption>Source: Microsoft<\/figcaption><\/figure><\/div>\n\n\n\n<ul class=\"wp-block-list\"><li>Firstly, data flows are created from the factory resources pane like pipelines and datasets.\u00a0<\/li><li>Secondly, \u00a0to create a data flow, select the plus sign next to\u00a0<strong>Factory Resources<\/strong>, and then select\u00a0<strong>Data Flow<\/strong>.<\/li><li>Thirdly, the action takes you to the data flow canvas, where you can create your transformation logic. <\/li><li>Then, select\u00a0<strong>Add source<\/strong>\u00a0to start configuring your source transformation.\u00a0<\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Elements of Data Flow<\/strong><\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>Authoring data flows<\/strong>: Mapping data flow has a unique authoring canvas designed to make building transformation logic easy.\u00a0<\/li><li><strong>Graph: <\/strong>The graph displays the transformation stream and displays the lineage of source data as it flows into one or more sinks.<\/li><li><strong>Configuration Panel: <\/strong>The configuration panel shows the settings specific to the currently selected transformation. If no transformation is selected, it shows the data flow.<\/li><li><strong>Transformation settings<\/strong>: The first tab in each transformation&#8217;s configuration pane contains the settings specific to that transformation.\u00a0<\/li><li><strong>Optimize: <\/strong>The\u00a0Optimize\u00a0tab contains settings to configure partitioning schemes.\u00a0 <\/li><li><strong>Inspect<\/strong>: The\u00a0Inspect\u00a0tab provides a view into the metadata of the data stream that is being transformed. Inspect\u00a0is a read-only view of your metadata. It is not required to have the debug mode enabled to see metadata in the\u00a0Inspect\u00a0pane.<\/li><li><strong>Data preview:<\/strong> In case the debug mode is on, the\u00a0Data Preview\u00a0tab provides you with an interactive snapshot of the data at each transform.<\/li><\/ul>\n\n\n\n<p>For more visit: <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/concepts-data-flow-overview\" target=\"_blank\" rel=\"noreferrer noopener\">Available Transformations<\/a><\/p>\n\n\n\n<div class=\"wp-block-image is-style-default\"><figure class=\"aligncenter size-large\"><a href=\"https:\/\/www.testpreptraining.ai\/analyzing-data-with-microsoft-power-bi-da-100-free-test\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" width=\"961\" height=\"150\" src=\"https:\/\/www.testpreptraining.ai\/tutorial\/wp-content\/uploads\/2021\/01\/Microsoft-Exam-DA-100-Free-Practice-Test.png\" alt=\"Microsoft Exam DA-100 Free Practice Test\" class=\"wp-image-31483\" srcset=\"https:\/\/www.testpreptraining.ai\/tutorial\/wp-content\/uploads\/2021\/01\/Microsoft-Exam-DA-100-Free-Practice-Test.png 961w, https:\/\/www.testpreptraining.ai\/tutorial\/wp-content\/uploads\/2021\/01\/Microsoft-Exam-DA-100-Free-Practice-Test-750x117.png 750w\" sizes=\"auto, (max-width: 961px) 100vw, 961px\" \/><\/a><\/figure><\/div>\n","protected":false},"excerpt":{"rendered":"<p>The process of mapping data flows refers to visually designed data transformations in the Azure Data Factory. Also, the process of data flow permits data engineers to develop data transformation logic without writing code. Thereby, the resulting data flow are executed as activities within Azure Data Factory pipelines that use scaled-out Apache Spark clusters. Some&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"footnotes":""},"categories":[174],"tags":[],"class_list":["post-31963","page","type-page","status-publish","hentry","category-microsoft-azure"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Using Data Flows | Microsoft Exam DA-100 | Testprep Training Tutorials<\/title>\n<meta name=\"description\" content=\"Boost your learning and get ready to qualify Microsoft Exam DA-100 with latest and updated practice test. Learn about mapping Data Flows Now!\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Using Data Flows | Microsoft Exam DA-100 | Testprep Training Tutorials\" \/>\n<meta property=\"og:description\" content=\"Boost your learning and get ready to qualify Microsoft Exam DA-100 with latest and updated practice test. Learn about mapping Data Flows Now!\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/\" \/>\n<meta property=\"og:site_name\" content=\"Testprep Training Tutorials\" \/>\n<meta property=\"article:modified_time\" content=\"2021-01-14T11:43:09+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/media\/data-flow\/new-data-flow.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/\",\"url\":\"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/\",\"name\":\"Using Data Flows | Microsoft Exam DA-100 | Testprep Training Tutorials\",\"isPartOf\":{\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#website\"},\"datePublished\":\"2021-01-14T11:43:08+00:00\",\"dateModified\":\"2021-01-14T11:43:09+00:00\",\"description\":\"Boost your learning and get ready to qualify Microsoft Exam DA-100 with latest and updated practice test. Learn about mapping Data Flows Now!\",\"breadcrumb\":{\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.testpreptraining.ai\/tutorial\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Using Data Flows\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#website\",\"url\":\"https:\/\/www.testpreptraining.ai\/tutorial\/\",\"name\":\"Testprep Training Tutorials\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.testpreptraining.ai\/tutorial\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#organization\",\"name\":\"Testprep Training\",\"url\":\"https:\/\/www.testpreptraining.ai\/tutorial\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.testpreptraining.com\/tutorial\/wp-content\/uploads\/2020\/07\/tpt-logo-6.png\",\"contentUrl\":\"https:\/\/www.testpreptraining.com\/tutorial\/wp-content\/uploads\/2020\/07\/tpt-logo-6.png\",\"width\":583,\"height\":153,\"caption\":\"Testprep Training\"},\"image\":{\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Using Data Flows | Microsoft Exam DA-100 | Testprep Training Tutorials","description":"Boost your learning and get ready to qualify Microsoft Exam DA-100 with latest and updated practice test. Learn about mapping Data Flows Now!","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/","og_locale":"en_US","og_type":"article","og_title":"Using Data Flows | Microsoft Exam DA-100 | Testprep Training Tutorials","og_description":"Boost your learning and get ready to qualify Microsoft Exam DA-100 with latest and updated practice test. Learn about mapping Data Flows Now!","og_url":"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/","og_site_name":"Testprep Training Tutorials","article_modified_time":"2021-01-14T11:43:09+00:00","og_image":[{"url":"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/media\/data-flow\/new-data-flow.png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/","url":"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/","name":"Using Data Flows | Microsoft Exam DA-100 | Testprep Training Tutorials","isPartOf":{"@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#website"},"datePublished":"2021-01-14T11:43:08+00:00","dateModified":"2021-01-14T11:43:09+00:00","description":"Boost your learning and get ready to qualify Microsoft Exam DA-100 with latest and updated practice test. Learn about mapping Data Flows Now!","breadcrumb":{"@id":"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.testpreptraining.ai\/tutorial\/using-data-flows\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.testpreptraining.ai\/tutorial\/"},{"@type":"ListItem","position":2,"name":"Using Data Flows"}]},{"@type":"WebSite","@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#website","url":"https:\/\/www.testpreptraining.ai\/tutorial\/","name":"Testprep Training Tutorials","description":"","publisher":{"@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.testpreptraining.ai\/tutorial\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#organization","name":"Testprep Training","url":"https:\/\/www.testpreptraining.ai\/tutorial\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#\/schema\/logo\/image\/","url":"https:\/\/www.testpreptraining.com\/tutorial\/wp-content\/uploads\/2020\/07\/tpt-logo-6.png","contentUrl":"https:\/\/www.testpreptraining.com\/tutorial\/wp-content\/uploads\/2020\/07\/tpt-logo-6.png","width":583,"height":153,"caption":"Testprep Training"},"image":{"@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/pages\/31963","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/comments?post=31963"}],"version-history":[{"count":2,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/pages\/31963\/revisions"}],"predecessor-version":[{"id":31977,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/pages\/31963\/revisions\/31977"}],"wp:attachment":[{"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/media?parent=31963"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/categories?post=31963"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/tags?post=31963"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}