mlfinlab容器化之旅

事情起因是我要用mlfinlab里面的函数。但是由于后期mlfinlab商业化，我想用免费的版本，只找到个mlfinlab-0.15.3-py3-none-any.whl安装包，但是由于久远，安装完之后，调用函数各种报错。

Python 3.7.15 (default, Nov 24 2022, 21:12:53)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type “help”, “copyright”, “credits” or “license” for more information.
>>> import mlfinlab
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
Traceback (most recent call last):
File “<stdin>”, line 1, in <module>
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/mlfinlab/__init__.py”, line 22, in <module>
import mlfinlab.portfolio_optimization as portfolio_optimization
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/mlfinlab/portfolio_optimization/__init__.py”, line 5, in <module>
from mlfinlab.portfolio_optimization.modern_portfolio_theory import CriticalLineAlgorithm
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/mlfinlab/portfolio_optimization/modern_portfolio_theory/__init__.py”, line 5, in <module>
from mlfinlab.portfolio_optimization.modern_portfolio_theory.mean_variance import MeanVarianceOptimisation
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/mlfinlab/portfolio_optimization/modern_portfolio_theory/mean_variance.py”, line 9, in <module>
import cvxpy as cp
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/cvxpy/__init__.py”, line 18, in <module>
from cvxpy.atoms import *
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/cvxpy/atoms/__init__.py”, line 20, in <module>
from cvxpy.atoms.geo_mean import geo_mean
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/cvxpy/atoms/geo_mean.py”, line 20, in <module>
from cvxpy.utilities.power_tools import (fracify, decompose, approx_error, lower_bound,
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/cvxpy/utilities/power_tools.py”, line 18, in <module>
from cvxpy.atoms.affine.reshape import reshape
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/cvxpy/atoms/affine/reshape.py”, line 18, in <module>
from cvxpy.atoms.affine.hstack import hstack
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/cvxpy/atoms/affine/hstack.py”, line 18, in <module>
from cvxpy.atoms.affine.affine_atom import AffAtom
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/cvxpy/atoms/affine/affine_atom.py”, line 22, in <module>
from cvxpy.cvxcore.python import canonInterface
File “/root/miniconda3_mlf_im/lib/python3.7/site-packages/cvxpy/cvxcore/python/__init__.py”, line 3, in <module>
import _cvxcore
ImportError: numpy.core.multiarray failed to import

或者numpy升级 导致函数无法使用，后来就想着这事安装环境，不如用docker纯粹。就想实现mlfinlab-0.15.3容器化。

这次主要是通过deepseek R1 和 google gemini 去实现的，就是提供安装文件，然后让它写dockerfile文件容器化。

测试下来deepseek R1没能完成目标，google gemini 最终完成了，耗时一周，真的是累死。

这也说明当前AI 中 google gemini 算领先了。

最终google gemini 提供的dockerfile文件：

# 1. 使用兼容的 Python 基础镜像
FROM python:3.8-slim

# 2. 安装系统级的编译工具
RUN apt-get update && \
apt-get install -y build-essential && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# 3. 设置工作目录
WORKDIR /app

# 4. 复制本地的 wheel 文件到镜像中
COPY mlfinlab-0.15.3-py3-none-any.whl .

# 5. 【最终方案】分步并强制使用全局环境来安装依赖
# 步骤 5a: 首先安装 numpy
RUN pip install numpy==1.18.5

# 步骤 5b: 安装其余所有依赖包
RUN pip install \
–no-build-isolation \
pandas==1.0.4 \
scipy==1.4.1 \
statsmodels==0.11.1 \
scikit-learn==0.23.1 \
matplotlib==3.2.1 \
seaborn \
numba==0.49.1 \
tensorflow==2.2.1 \
cvxpy==1.1.1 \
analytics-python==1.2.9 \
getmac==0.8.2 \
POT==0.7.0 \
dash==1.14.0 \
dash-bootstrap-components==0.10.3 \
dash-core-components==1.10.2 \
dash-html-components==1.0.3 \
dash-table==4.9.0 \
jupyter-dash==0.3.1 \
dash-cytoscape==0.2.0 \
Werkzeug==2.0.3 \
networkx==2.5

# 6. 使用 –no-deps 安装 mlfinlab
RUN pip install –no-deps mlfinlab-0.15.3-py3-none-any.whl

# 7. 【终极修正】使用 sed 命令对已安装的 mlfinlab 代码进行永久性修复
# 一次性注释掉 __init__.py 文件中所有已知的、有问题的遥测调用
RUN sed -i -e “s/devadarsh.page(‘Import’)/# devadarsh.page(‘Import’)/g” \
-e “s/devadarsh.identify()/# devadarsh.identify()/g” \
/usr/local/lib/python3.8/site-packages/mlfinlab/__init__.py

# 8. 使用独立的脚本文件进行验证
COPY verify.py .
RUN python3 verify.py

# 9. (可选) 安装 Jupyter Lab
RUN pip install jupyterlab

# 10. (可选) 暴露端口
EXPOSE 8888

# 11. (可选) 设置启动命令
CMD [“jupyter”, “lab”, “–ip=0.0.0.0”, “–port=8888”, “–no-browser”, “–allow-root”]

另外需要单独verify.py 的文件。

verify.py如下：

# verify.py

# 导入所有核心库
import numpy as np
import scipy
import mlfinlab

# — 开始验证 —
print(‘—— Verification Start ——‘)
print(f’NumPy version: {np.__version__}’)
print(f’SciPy version: {scipy.__version__}’)
# mlfinlab 库没有 __version__ 属性，但 import 成功即代表安装成功
print(‘MLFinLab module imported successfully!’)
print(‘—— Verification End: All looks good! Build successful! ——‘)

 

最终测试mlfinlab没有报错，mlfinlab 我需要的函数也正常使用，完美。
最后就主机下载最新python:3.8（老版本的python3.8 会异常报错：python3.8/site-packages/_cvxcore.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZSt28__throw_bad_array_new_lengthv 最好下载最新的版本，就没问题），按照步骤重新安装到本地，完美解决问题。
后续继续测试。完美！