BEGIN:VCALENDAR PRODID;X-RICAL-TZSOURCE=TZINFO:-//Calagator//EN CALSCALE:GREGORIAN X-WR-CALNAME:Calagator METHOD:PUBLISH VERSION:2.0 BEGIN:VTIMEZONE TZID;X-RICAL-TZSOURCE=TZINFO:America/Los_Angeles BEGIN:DAYLIGHT DTSTART:20250309T020000 RDATE:20250309T020000 TZOFFSETFROM:-0800 TZOFFSETTO:-0700 TZNAME:PDT END:DAYLIGHT END:VTIMEZONE BEGIN:VEVENT CREATED;VALUE=DATE-TIME:20250123T182324Z DTEND;TZID=America/Los_Angeles;VALUE=DATE-TIME:20250415T190000 DTSTART;TZID=America/Los_Angeles;VALUE=DATE-TIME:20250415T170000 DTSTAMP;VALUE=DATE-TIME:20250123T182324Z LAST-MODIFIED;VALUE=DATE-TIME:20250123T182324Z UID:http://calagator.org/events/1250481744 DESCRIPTION:The Portland Azure User Group is a community of people excite d about the endless possibilities of cloud computing\, data analytics\, and security on Microsoft Azure. We believe the Azure platform offers ri ch possibilities. We keep pace by sharing our learning and doing deep-di ve explorations of topics as a community of learners. We run the gamut f rom new to experienced users and welcome anyone who shares an interest i n cloud scalable infrastructure\, applications\, tools\, analytics\, ide ntity management\, and security. \;\n \;\nSome great perks to join ing our monthly meetups are networking\, knowledge transfer\, and commun ity building. We aim to foster a vibrant community that shares success i n implementing best practices for Microsoft Azure. \;\n \;\nSessio n Details: Metadata Driven Pipelines for Dynamic Full and Incremental Pr ocessing in Azure SQL \;\n \;\nDeveloping ETLs/ELTs can be a compl ex process when you add in business logic\, large amounts of data\, and the high volume of table data that needs to be moved from source to targ et. This is especially true in analytical workloads involving Azure SQL when there is a need to either fully reload a table or incrementally upd ate a table. In order to handle the logic to incrementally update a tabl e or fully reload a table in Azure SQL (or Azure Synapse)\, we will need to create the following assets: \;\n• Metadata table in Azure SQL 3\;\n◦ This will contain the configurations needed to load each table en d to end \;\n• Metadata driven pipelines \;\n◦ Parent and child pi peline templates that will orchestrate and execute the ETL/ELT end to en d \;\n• Custom SQL logic for incremental processing \;\n◦ Dynamic SQL to perform the delete and insert based on criteria the user provides in the metadata table \;\n \;\nThis session uses Azure SQL DB as the source and sink databases. However\, Azure SQL MI\, On-Prem SQL\, an d Synapse Dedicated Pools (along with Synapse Pipelines) will also be co mpatible for this solution. As a source\, you can use databases like MyS QL\, Oracle\, and others. You will just need to adjust the query syntax/ connections to match the desired source. \;\n \;\nGuest Speaker: M arc Bushong \;\nMarc Bushong\, Senior Cloud Solution Architect at Mic rosoft\, has extensive experience in healthcare and higher education. He excels in Azure architecture\, end-to-end solution design\, and full-st ack Azure development. His career began as a Radiologic Technologist\, p roviding him with a unique perspective on the healthcare industry\, from direct patient care to backend data engineering processes.\n\nTags: .ne t\, asp.net\, cloud computing\, containers\, docker\, Enterprise Archite cture\, iaas (infrastructure as a service)\, it professionals\, kubernet es\, linux\, machine learning\, Microsoft Azure\, serverless architectur e\, serverless computing\, software development\, azure\n\nImported from : http://calagator.org/events/1250481744 URL:https://www.meetup.com/portland-azure-user-group/events/305236488/?ut m_medium=referral&utm_campaign=share-btn_savedevents_share_modal&utm_sou rce=link SUMMARY:Portland Azure User Group LOCATION:Virtual Event - Microsoft Teams (Register to access event): fals e SEQUENCE:1 END:VEVENT END:VCALENDAR