4.0 ML Solution Monitoring, Maintenance, and Security MLA-C01 Practice Quiz

• A. Model drift involves changes in model behavior or data relationships that can reduce prediction quality after deployment.
• B. Data drift involves changes in the distribution of production input data compared with training or baseline data.
• C. Data quality monitoring involves tracking production data for missing values, schema changes, invalid values, or other quality issues that can affect inference.
• D. Model performance monitoring involves tracking production prediction quality or business metrics to determine whether a model continues to meet requirements.

Model drift involves changes in model behavior or data relationships that can reduce prediction quality after deployment. This is the correct answer.

• A. Infrastructure utilization involves tracking how much CPU, memory, GPU, disk, or network capacity ML infrastructure consumes.
• B. Data drift involves changes in the distribution of production input data compared with training or baseline data.
• C. Throughput metric involves measuring how many requests, records, or jobs an ML system processes over time.
• D. Availability metric involves measuring whether an ML system or endpoint remains reachable and operational when needed.

Data drift involves changes in the distribution of production input data compared with training or baseline data. This is the correct answer.

• A. CloudWatch alarms is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: CloudWatch alarms involves triggering notifications or automated actions when monitored metrics cross configured thresholds.
• B. Model drift is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: Model drift involves changes in model behavior or data relationships that can reduce prediction quality after deployment.
• C. CloudWatch dashboards is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: CloudWatch dashboards involves visualizing key ML infrastructure and application metrics in a centralized operational view.
• D. AWS X-Ray tracing is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: AWS X-Ray tracing involves tracing requests through distributed applications to investigate latency and performance bottlenecks.

Model drift involves changes in model behavior or data relationships that can reduce prediction quality after deployment. This matches the ML monitoring, maintenance, and security scenario without confusing inference monitoring, infrastructure metrics, observability, cost tools, IAM, encryption, network isolation, or pipeline security. This is the correct answer.

• A. AWS Cost Explorer is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: AWS Cost Explorer involves analyzing and visualizing AWS cost and usage trends over time.
• B. AWS Budgets is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: AWS Budgets involves tracking cost or usage thresholds and sending alerts when actual or forecasted values exceed limits.
• C. Data drift is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: Data drift involves changes in the distribution of production input data compared with training or baseline data.
• D. Spot Instances for ML cost is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: Spot Instances for ML cost involves using spare EC2 capacity at lower cost for interruption-tolerant ML workloads.

Data drift involves changes in the distribution of production input data compared with training or baseline data. This matches the ML monitoring, maintenance, and security scenario without confusing inference monitoring, infrastructure metrics, observability, cost tools, IAM, encryption, network isolation, or pipeline security. This is the correct answer.

• A. Bucket policies for ML artifacts is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: Bucket policies for ML artifacts involves controlling access to S3 buckets that store datasets, model artifacts, logs, or pipeline outputs.
• B. AWS KMS encryption is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: AWS KMS encryption involves protecting ML datasets, model artifacts, volumes, logs, and outputs with managed cryptographic keys.
• C. VPC isolation for ML systems is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: VPC isolation for ML systems involves placing ML resources in controlled VPC subnets and network paths to reduce exposure.
• D. Data quality monitoring is the best fit when an ML monitoring, maintenance, or security workflow requires this behavior: Data quality monitoring involves tracking production data for missing values, schema changes, invalid values, or other quality issues that can affect inference.

Data quality monitoring involves tracking production data for missing values, schema changes, invalid values, or other quality issues that can affect inference. This matches the ML monitoring, maintenance, and security scenario without confusing inference monitoring, infrastructure metrics, observability, cost tools, IAM, encryption, network isolation, or pipeline security. This is the correct answer.

• A. A workflow points to IAM roles for ML applications when inference monitoring, infrastructure optimization, cost control, audit logging, or security depends on this distinction: IAM roles for ML applications involves providing temporary permissions for applications, notebooks, pipelines, training jobs, or endpoints that access AWS resources.
• B. A workflow points to IAM policies for ML systems when inference monitoring, infrastructure optimization, cost control, audit logging, or security depends on this distinction: IAM policies for ML systems involves defining allowed or denied actions for users, services, applications, and ML resources.
• C. A workflow points to Model drift when inference monitoring, infrastructure optimization, cost control, audit logging, or security depends on this distinction: Model drift involves changes in model behavior or data relationships that can reduce prediction quality after deployment.
• D. A workflow points to SageMaker Role Manager when inference monitoring, infrastructure optimization, cost control, audit logging, or security depends on this distinction: SageMaker Role Manager involves helping create persona-based IAM roles for SageMaker users and ML activities.

Model drift involves changes in model behavior or data relationships that can reduce prediction quality after deployment. This distinction matters because choosing a nearby concept would lead to the wrong drift signal, operational metric, log source, cost-control method, access policy, encryption control, or network protection. This is the correct answer.

• A. A workflow points to Model drift when inference monitoring, infrastructure optimization, cost control, audit logging, or security depends on this distinction: Model drift involves changes in model behavior or data relationships that can reduce prediction quality after deployment.
• B. A workflow points to Data quality monitoring when inference monitoring, infrastructure optimization, cost control, audit logging, or security depends on this distinction: Data quality monitoring involves tracking production data for missing values, schema changes, invalid values, or other quality issues that can affect inference.
• C. A workflow points to Model performance monitoring when inference monitoring, infrastructure optimization, cost control, audit logging, or security depends on this distinction: Model performance monitoring involves tracking production prediction quality or business metrics to determine whether a model continues to meet requirements.
• D. A workflow points to Data drift when inference monitoring, infrastructure optimization, cost control, audit logging, or security depends on this distinction: Data drift involves changes in the distribution of production input data compared with training or baseline data.

Data drift involves changes in the distribution of production input data compared with training or baseline data. This distinction matters because choosing a nearby concept would lead to the wrong drift signal, operational metric, log source, cost-control method, access policy, encryption control, or network protection. This is the correct answer.

• A. SageMaker Clarify drift detection is the correct ML engineering choice when the monitoring, maintenance, or security decision depends on this exact behavior: SageMaker Clarify drift detection involves detecting changes related to bias or feature attribution that can affect model behavior over time.
• B. A/B testing for models is the correct ML engineering choice when the monitoring, maintenance, or security decision depends on this exact behavior: A/B testing for models involves comparing model variants in production by sending portions of traffic to different variants and measuring performance.
• C. Production variant is the correct ML engineering choice when the monitoring, maintenance, or security decision depends on this exact behavior: Production variant involves a deployed model version or endpoint configuration that receives a defined portion of inference traffic.
• D. Model drift is the correct ML engineering choice when the monitoring, maintenance, or security decision depends on this exact behavior: Model drift involves changes in model behavior or data relationships that can reduce prediction quality after deployment.

Model drift involves changes in model behavior or data relationships that can reduce prediction quality after deployment. This applies the concept at the point where an ML engineer must keep ML solutions reliable, cost-aware, observable, and secure in production. This is the correct answer.