Apple Silicon Experiment 6— MLX Framework BERT Port [WIP]

Apple announced MLX Framework for ML Models

GitHub - ml-explore/mlx: MLX: An array framework for Apple silicon

Porting transformers package BERT model for MLX inference

Environment

• transformers 4.33.2
• mlx 0.0.3
• M1 Max — macOS 14.1.2 with Python 3.8
• model: bert-base-uncased from huggingface

Loading Model Example:

Weight Conversion:

Inference Example:

• sequence_output, pooled_output: results from the MLX model
• og_model_outputs: results from transformers package model

Code:

GitHub - id4thomas/mlx_usage: trying out mlx framework

References:

• https://github.com/huggingface/transformers/blob/main/src/transformers/models/bert/modeling_bert.py
• https://ml-explore.github.io/mlx/build/html/index.html