Snowflake DSA-C03 Exam : SnowPro Advanced: Data Scientist Certification Exam

You have deployed a regression model in Snowflake as an external function using AWS Lambda'. The external function takes several numerical features as input and returns a predicted value. You want to continuously monitor the model's performance in production and automatically retrain it when the performance degrades below a predefined threshold. Which of the following methods represent VALID approaches for calculating and monitoring model performance within the Snowflake environment and triggering the retraining process?

• A. Create a Snowflake Task that periodically executes a SQL query to calculate performance metrics (e.g., RMSE) by comparing predicted values from the external function with actual values stored in a separate table. Trigger a Python UDF, deployed as a Snowflake stored procedure, to initiate retraining if the RMSE exceeds the threshold.
• B. Create a view that joins the input features with the predicted output and the actual result. Configure model monitoring within the AWS Sagemaker to perform continuous validation of the model.
• C. Utilize Snowflake's Alerting feature, setting an alert rule based on the output of a SQL query that calculates performance metrics. Configure the alert action to invoke a webhook that triggers a retraining pipeline.
• D. Implement custom logging within the AWS Lambda function to capture prediction results and actual values. Configure AWS CloudWatch to monitor these logs and trigger an AWS Step Function that initiates a new training job and updates the Snowflake external function with the new model endpoint upon completion.
• E. Build a Snowpark Python application deployed on Snowflake which periodically polls the external function's performance by querying the function with a sample data set and comparing results to ground truth stored in Snowflake. Initiate retraining directly from the Snowpark application if performance degrades.

You are validating a time series forecasting model for daily sales using Snowflake and Snowpark. The residuals plot shows a clear sinusoidal pattern. Which of the following actions should you consider to improve your model? (Select all that apply)

• A. Increase the regularization strength in your model.
• B. Change the algorithm to a linear regression model, since it is more likely to capture sinusoidal patterns
• C. Remove outlier data points to improve overall model performance.
• D. Incorporate lagged features representing previous sales values (e.g., sales from the previous day, week, or month).
• E. Apply a Box-Cox transformation to the target variable (sales) to stabilize the variance.

A pharmaceutical company is testing a new drug to lower blood pressure. They conduct a clinical trial with 200 patients. After treatment, the sample mean reduction in systolic blood pressure is 10 mmHg, with a sample standard deviation of 15 mmHg. You want to construct a 99% confidence interval for the true mean reduction in systolic blood pressure. Which of the following statements is most accurate concerning the appropriate distribution and critical value to use?

• A. Use a z-distribution because the sample size is large (n > 30), and the critical value is approximately 2.576.
• B. Use a z-distribution because we are estimating mean, and use a critical value of 1.96.
• C. Use a t-distribution with 199 degrees of freedom, and the critical value is slightly larger than 2.576.
• D. Use a t-distribution with 200 degrees of freedom, and the critical value is close to 2.576.
• E. Use a chi-squared distribution with 199 degrees of freedom.

You have deployed a vectorized Python UDF in Snowflake to perform sentiment analysis on customer reviews. The UDF uses a pre-trained transformer model loaded from a Stage. The model consumes a significant amount of memory (e.g., 5GB). Users are reporting intermittent 'Out of Memory' errors when calling the UDF, especially during peak usage. Which of the following strategies, used IN COMBINATION, would MOST effectively mitigate these errors and optimize resource utilization?

• A. Increase the value of 'MAX BATCH_ROWS' for the UDF to process larger batches of data at once.
• B. Reduce the value of 'MAX for the UDF to process smaller batches of data.
• C. Implement lazy loading of the model within the UDF, ensuring it's only loaded once per warehouse node and reused across multiple invocations within that node.
• D. Increase the warehouse size to provide more memory per node.
• E. Partition the input data into smaller chunks using SQL queries and call the UDF on each partition separately.

You are responsible for deploying a fraud detection model in Snowflake. The model needs to be validated rigorously before being put into production. Which of the following actions represent the MOST comprehensive approach to model validation within the Snowflake environment, focusing on both statistical performance and operational readiness, and using Snowflake features for validation?

• A. Performing a single train/test split of the historical data and evaluating model performance metrics (e.g., accuracy, precision, recall) on the test set using standard Python libraries within a Snowflake Snowpark environment. Deploying the model directly if the metrics exceed a predefined threshold.
• B. Implementing K-fold cross-validation using Snowflake stored procedures and temporary tables to store and aggregate the results from each fold. Evaluating the model's performance across different data segments and time periods to assess its robustness. Using Snowflake streams and tasks to automate the validation process on new incoming data.
• C. Relying on a simple visual inspection of model outputs and comparing them to a small sample of known fraud cases. Skipping formal validation to accelerate the deployment process.
• D. Conducting a comprehensive backtesting analysis using historical data, simulating real-world scenarios, and evaluating the model's performance under different conditions. Using Snowflake's time travel feature to access historical data snapshots for accurate backtesting. Monitoring model performance using Snowflake alerts triggered by custom SQL queries against model prediction logs.
• E. Calculating only the AUC (Area Under the Curve) metric on the entire dataset without performing any data splitting or cross-validation. Deploying the model if the AUC is above 0.7.