Databricks-Machine-Learning-Professional Databricks Certified Machine Learning Professional Questions and Answers

The DESCRIBE HISTORY command can be used to view the commit history of a Delta table, including the schema changes, operations, and timestamps. This command can help identify when a column was dropped from the feature set and by which operation. The other commands are either invalid or do not provide the required information.  References :

Delta Lake - View Commit History

Databricks Certified Machine Learning Professional Exam Guide  - Section 1: Experimentation - Data Management

The correct line of code is “MODEL_VERSION_TRANSITIONED_STAGE”, which is the name of the event that occurs when a model version transitions to any stage in the MLflow Model Registry. This event can be used to trigger a webhook that will execute a Databricks Job based on the model version and the stage it transitions to. The other options are either invalid event names or too specific for the task.

References:   MLflow Model Registry Webhooks ,  MLflow Model Registry Events

 To read data from a stream source, such as Kafka, socket, or rate, the  spark.readStream  method should be used instead of  spark.read . The  spark.readStream  method returns a streaming DataFrame that represents the unbounded input data stream. The  spark.readStream  method supports the same options and formats as the  spark.read  method, such as schema, delta, csv, json, etc. The  spark.readStream  method can also read from a Delta table as a stream source, by specifying the  format("delta")  and the path or table name of the Delta table 1 2 3

The other options are incorrect because:

A. Replacing “inference” with the path to the location of the Delta table does not change the fact that  spark.read  is used to read from a stream source, which is not supported. The  spark.readStream  method should be used instead, and the path or table name of the Delta table can be specified as an option or argument.

B. Replacing  schema(schema)  with  option("maxFilesPerTrigger", 1)  does not change the fact that  spark.read  is used to read from a stream source, which is not supported. The  spark.readStream  method should be used instead, and the schema can be specified as an option or argument. The  option("maxFilesPerTrigger", 1)  is an optional configuration that limits the number of files processed in each trigger for file-based stream sources, such as delta, csv, json, etc.  It does not affect the reading of data from a stream source 4

D. Replacing  format("delta")  with  format("stream")  does not change the fact that  spark.read  is used to read from a stream source, which is not supported. The  spark.readStream  method should be used instead, and the format can be specified as an option or argument. The  format("stream")  is not a valid format for reading data from a stream source.  The supported formats are delta, kafka, socket, rate, etc 1

E. Replacing  predict  with a stream-friendly prediction function does not change the fact that  spark.read  is used to read from a stream source, which is not supported. The  spark.readStream  method should be used instead, and the prediction function can be applied to the streaming DataFrame as usual. The  predict  function does not need to be changed, as long as it can accept a streaming DataFrame as input and return a column of predictions as output 5

References :

Input Sources - Structured Streaming Programming Guide - Spark 3.2.0 Documentation

Structured Streaming + Delta Lake - Databricks

Structured Streaming Programming Guide - Spark 3.2.0 Documentation

Configuration - Structured Streaming Programming Guide - Spark 3.2.0 Documentation

Machine Learning with Structured Streaming - Databricks

MLflow Autologging is a feature that automatically logs parameters, metrics, and artifacts for machine learning frameworks, such as Hyperopt, without requiring any code changes. When using MLflow Autologging with Hyperopt, MLflow will automatically create a parent run for the hyperparameter tuning process and a child run for each unique combination of hyperparameter values. This way, the user can easily compare and visualize the results of different hyperparameter settings in the MLflow UI. MLflow Autologging also supports nested runs, which means that if the user starts a manual run before calling fmin, the parent run will be nested under the manual run. This allows the user to organize and group related runs together.  References:   MLflow Autologging ,  Hyperparameter Tuning with MLflow, Apache Spark MLlib and Hyperopt ,  Using MLFlow with Hyperopt for Automated Machine Learning ,  Parallelize hyperparameter tuning with scikit-learn and MLflow , [Optimizing Model Performance with MLflow and Hyperopt: From Hyperparameter Tuning to Serving]

Feature drift is a change in the distribution of the input data over time, which can affect the performance and accuracy of the model. Monitoring feature drift is important to ensure that the model is still valid and reliable for the current data. One simple and low-cost method of monitoring numeric feature drift is to track the summary statistics trends of the input features, such as mean, standard deviation, minimum, maximum, etc. These statistics can be computed easily and efficiently, and can provide a quick overview of the changes in the data distribution. If the summary statistics deviate significantly from the baseline values, it may indicate a feature drift. However, summary statistics trends may not capture all the nuances of the data distribution, such as outliers, multimodality, skewness, etc.  Therefore, other methods, such as statistical tests or visualizations, may be needed to complement the summary statistics trends and provide a more comprehensive analysis of the feature drift 1 2 3   References :

Monitoring Feature Drift - Databricks

Monitor feature attribution skew and drift | Vertex AI | Google Cloud

Monitoring Model Drift - 6 Different Methodologies - Qualdo

 The mlflow.register_model function takes two arguments: model_uri and name. The model_uri is the URI of the model that was logged to MLflow, which can be obtained from the best run object. The name is the name of the registered model in the MLflow Model Registry. Therefore, the correct line of code to register the model is:

mlflow.register_model(model_uri, “best_model”)

This will create a new registered model with the name “best_model” and register the model version from the best run as the first version of that model.

References:

[mlflow.register_model — MLflow 1.22.0 documentation]

[MLflow Model Registry — Databricks Documentation]

[Manage MLflow Models — Databricks Documentation] Message has links.

The code block that the team is using is correct and does not need any changes to accomplish the task. The  update_registered_model  method of the  MlflowClient  class can be used to update the metadata of a registered model, such as its name or description. The method takes the following parameters:

name: The name of the registered model to update.

description: The new description for the registered model.

new_name: The new name for the registered model.

The method returns a  RegisteredModel  object that represents the updated registered model 1

The other options are incorrect because:

A. Replacing  update_registered_model  with  update_model_version  would not update the metadata of the registered model, but rather the metadata of a specific model version. The  update_model_version  method can be used to update the stage, description, or name of a model version. The method takes the following parameters:

name: The name of the registered model that contains the model version to update.

version: The version number of the model version to update.

stage: The new stage for the model version.

description: The new description for the model version.

archive_existing_versions: Whether to archive other model versions in the same stage.

new_name: The new name for the model version.

The method returns a  ModelVersion  object that represents the updated model version 2

C. Replacing  description  with  artifact  would not update the description of the registered model, but rather raise an error, as  artifact  is not a valid parameter for the  update_registered_model  method. The  artifact  parameter is used for the  log_model  or  save_model  methods, which can be used to log or save a model with a specific flavor and artifact path 3 4

D. Replacing  client.update_registered_model  with  mlflow  would not update the registered model, but rather raise an error, as  mlflow  is not a valid method, but rather a module that contains various submodules and functions. The  mlflow  module does not have an  update_registered_model  function, but rather the  MlflowClient  class has an  update_registered_model  method 5

E. Adding a Python model as an argument to  update_registered_model  would not update the registered model, but rather raise an error, as a Python model is not a valid argument for the  update_registered_model  method. The  update_registered_model  method does not take a model as an argument, but rather the name and description of the registered model to update. To add a Python model to the Model Registry, the  create_model_version  method of the  MlflowClient  class can be used, which takes the following parameters:

name: The name of the registered model to which to add the model version.

source: The source of the model, such as a local path, a run ID, or a model URI.

run_id: The run ID that generated the model, if any.

tags: A dictionary of key-value pairs to tag the model version with.

description: A description for the model version.

The method returns a  ModelVersion  object that represents the created model version 6

References :

mlflow.tracking.client.MlflowClient.update_registered_model — MLflow 2.9.1 documentation

mlflow.tracking.client.MlflowClient.update_model_version — MLflow 2.9.1 documentation

mlflow. < model_flavor > .log_model — MLflow 2.9.1 documentation

mlflow. < model_flavor > .save_model — MLflow 2.9.1 documentation

mlflow — MLflow 2.9.1 documentation

mlflow.tracking.client.MlflowClient.create_model_version — MLflow 2.9.1 documentation

Real-time deployment is a paradigm that can centrally compute predictions for a single record with exceedingly fast results. Real-time deployment involves exposing a model as a REST API endpoint that can receive requests and return predictions in milliseconds. This paradigm is suitable for scenarios where low latency and high throughput are required, such as fraud detection, recommendation systems, or chatbots. The other deployment paradigms are not as fast or as centralized as real-time deployment. Streaming deployment involves processing data streams in micro-batches, which can introduce some delay. Batch deployment involves processing large batches of data at scheduled intervals, which can take hours or days. Edge or on-device deployment involves deploying models to edge devices, such as mobile phones or IoT sensors, which can reduce the network overhead but also limit the computational power and model size.  https://docs.databricks.com/en/machine-learning/ml-tutorials.html

Real-time deployment is a type of model deployment where predictions are computed on-demand at query time. This can be useful when the feature values are not available until the time of delivery, and results are needed exceedingly fast for one record at a time.  Real-time deployment can leverage Databricks Model Serving, which allows users to host MLflow models as REST endpoints on Databricks 1 . The other options are incorrect because they either do not meet the requirements or are not supported by Databricks Machine Learning. Edge/on-device deployment is a type of model deployment where predictions are computed on the edge device, such as a smartphone or a sensor. This can be useful when there is limited or no network connectivity, or when privacy and security are important.  However, edge deployment is not supported by Databricks Machine Learning 2 . Streaming deployment is a type of model deployment where predictions are computed on streaming data, such as sensor readings or web logs. This can be useful when the data volume is high and the results need to be updated continuously.  However, streaming deployment is not supported by Databricks Model Serving 1 . Batch deployment is a type of model deployment where predictions are computed in advance and stored for later use. This can be useful when the feature values are known ahead of time and the predictions do not need to be delivered quickly. However, batch deployment does not meet the requirements of this scenario, as the feature values are not available until the time of delivery, and results are needed exceedingly fast for one record at a time.  References:   MLflow Model Serving ,  Model deployment patterns

Label drift is a type of data drift that occurs when there is a change in the probability of a label P(Y). This means that the output distribution of the model’s data source has changed over time, which can affect the performance and accuracy of the model. For example, if a model is trained to classify images of animals, and the data source starts to provide more images of cats than dogs, this would be an example of label drift. Label drift can be caused by various factors, such as changes in user behavior, environmental conditions, market trends, etc.  Label drift can be detected and handled by various methods, such as periodic retraining, online learning, ensemble methods, etc 1 2 3   References :

Monitoring Feature Drift - Databricks

Drift Metrics: How to Select the Right Metric to Analyze Drift

Detect data drift on datasets (preview) - Azure Machine Learning

The Databricks REST APIs allow users to programmatically access Databricks features and services, such as clusters, jobs, workspaces, libraries, secrets, and more. The Jobs API enables users to create, edit, delete, run, and list jobs, as well as manage job runs. Users can use the Jobs API to programmatically create a new Databricks job whose schedule depends on the result of some automated tests in a machine learning pipeline. For example, users can use the  create  endpoint to create a new job with a  cron_schedule  parameter that specifies the cron expression for the job schedule. Users can also use the  run_now  endpoint to run an existing job with a  run_name  parameter that specifies the name of the run. Users can use the  runs/get  endpoint to get the run result state of a job run, and use conditional logic to trigger another job run based on the result. Users can also use the  runs/list  endpoint to list all the runs of a job, and use the  runs/cancel  endpoint to cancel a job run if needed.  References :

Databricks REST API — Databricks Documentation , p. 1

Jobs API — Databricks Documentation , p. 1-2

Create and run Databricks Jobs | Databricks on AWS , Create and run jobs using the CLI, API, or notebooks, p. 1

To move a model version from the Staging stage to the Production stage using the MLflow Client, you can use the client.transition_model_version_stage method. This method takes the name of the registered model, the version of the model, and the target stage as arguments. It transitions the model version to the specified stage and returns the updated model version object. The syntax of the method is as follows:

Python

client.transition_model_version_stage (name, version, stage)

AI-generated code. Review and use carefully.  More info on FAQ .

The other options are incorrect because:

Option A: client.transition_model_version_stage with the stage argument set to “Staging” will not move the model version from the Staging stage to the Production stage, but rather keep it in the same stage.

Option B: client.transition_model_stage is not a valid method name, as it is missing an underscore. The correct method name is client.transition_model_version_stage.

Option D: client.transition_model_stage is not a valid method name, as it is missing an underscore. The correct method name is client.transition_model_version_stage. Also, the method does not take from and to arguments, but only a stage argument.

Option E: client.transition_model_version_stage does not take from and to arguments, but only a stage argument. The from argument is redundant, as the method will automatically transition the model version from its current stage to the target stage.  References :  MLflow Model Registry ,  MlflowClient.transition_model_version_stage

To log single values like the number of trees in a random forest, you can use the mlflow.log_param function. This function allows you to log a parameter (e.g. model hyperparameter) under the current run. Parameters can be of any type, and can be logged using the following syntax:

Python

mlflow.log_param("num_trees", 100)

AI-generated code. Review and use carefully.  More info on FAQ .

MLflow also offers a convenient way to log multiple parameters by indicating all of them using a dictionary 1 . The other options are incorrect because:

Option A: mlflow.log_artifact is used to log output files in any format, not single values 2 .

Option B: mlflow.log_model is used to log an MLflow Model along with its artifacts, not single values 3 .

Option C: mlflow.log_metric is used to log numeric metrics, not single values 4 .

Option E: There is a way to store values like this using the mlflow.log_param function 1 .  References :  mlflow.log_param ,  mlflow.log_artifact ,  mlflow.log_model ,  mlflow.log_metric

MLflow provides a module called  mlflow.pyfunc  that can be used to deploy any Python model as a PySpark UDF. The  mlflow.pyfunc.spark_udf  function takes a Spark session and a model URI, and returns a Spark UDF that can be applied to a Spark DataFrame. The model URI can be a local path, a remote path, or a run ID with a relative path within the MLflow tracking server. The  mlflow.pyfunc.spark_udf  function can be used with any model that has a  predict  function, such as scikit-learn, xgboost, etc 1 2

The other code blocks are incorrect because:

A. The  mlflow.pyfunc.spark_udf  function requires a Spark session as the first argument, not a Spark DataFrame.

B. It is possible to deploy a scikit-learn model on a Spark DataFrame using  mlflow.pyfunc.spark_udf  or other methods, such as broadcasting the model or using Fugue.

C. There is no module called  sklearn.spark_udf . To use a scikit-learn model as a Spark UDF,  mlflow.pyfunc.spark_udf  or other methods should be used.

D. There is no method called  spark.spark_udf . To create a Spark UDF, the  spark.udf.register  or  pyspark.sql.functions.udf  methods should be used.

References :

mlflow.pyfunc — MLflow 2.9.1 documentation

Deploy models for batch inference and prediction - Databricks

Concept drift is a change in the relationship between input and output data over time, which can affect the performance and accuracy of the model. To monitor concept drift, one needs to compare the predicted values from the model with the observed (actual) label values from the data source. A common way to do this is to compute an evaluation metric, such as accuracy, precision, recall, F1-score, etc., using the observed and predicted values. This metric can indicate how well the model is performing on the current data, and how it differs from the baseline performance on the training data.  If the metric shows a significant decline over time, it may suggest that concept drift has occurred and the model needs to be updated 1 2 3   References :

Monitoring Concept Drift - Databricks

Data drift, concept drift, and how to monitor for them - Mona Labs

8 Concept Drift Detection Methods To Use With ML Models - Aporia