Cat vs. Dog Classifier

1 Setup and Training

The first step is to install necessary dependencies and setup my environment. In my case the fastai toolbox as well as a search engine API. In chapter 2, bing search was the one used. Alas, that got deprecated in the meantime, so I am going to use duckduckgo search another free search API.

# Clean up previous folders
!rm -rf animal

# Install latest fastai + fastcore + fastbook
!pip install -U fastai fastcore fastbook ddgs

import fastbook
fastbook.setup_book()

Requirement already satisfied: fastai in /usr/local/lib/python3.11/dist-packages (2.7.19)
Collecting fastai
  Downloading fastai-2.8.2-py3-none-any.whl.metadata (9.5 kB)
Requirement already satisfied: fastcore in /usr/local/lib/python3.11/dist-packages (1.7.29)
Collecting fastcore
  Downloading fastcore-1.8.5-py3-none-any.whl.metadata (3.7 kB)
Collecting fastbook
  Downloading fastbook-0.0.29-py3-none-any.whl.metadata (13 kB)
Collecting ddgs
  Downloading ddgs-9.3.1-py3-none-any.whl.metadata (15 kB)
Requirement already satisfied: pip in /usr/local/lib/python3.11/dist-packages (from fastai) (24.1.2)
Requirement already satisfied: packaging in /usr/local/lib/python3.11/dist-packages (from fastai) (25.0)
Requirement already satisfied: fastdownload<2,>=0.0.5 in /usr/local/lib/python3.11/dist-packages (from fastai) (0.0.7)
Collecting fasttransform>=0.0.2 (from fastai)
  Downloading fasttransform-0.0.2-py3-none-any.whl.metadata (7.6 kB)
Requirement already satisfied: torchvision>=0.11 in /usr/local/lib/python3.11/dist-packages (from fastai) (0.21.0+cu124)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.11/dist-packages (from fastai) (3.10.0)
Requirement already satisfied: pandas in /usr/local/lib/python3.11/dist-packages (from fastai) (2.2.2)
Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from fastai) (2.32.3)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.11/dist-packages (from fastai) (6.0.2)
Requirement already satisfied: fastprogress>=0.2.4 in /usr/local/lib/python3.11/dist-packages (from fastai) (1.0.3)
Requirement already satisfied: pillow>=9.0.0 in /usr/local/lib/python3.11/dist-packages (from fastai) (11.2.1)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.11/dist-packages (from fastai) (1.6.1)
Requirement already satisfied: scipy in /usr/local/lib/python3.11/dist-packages (from fastai) (1.15.3)
Requirement already satisfied: spacy<4 in /usr/local/lib/python3.11/dist-packages (from fastai) (3.8.7)
Collecting plum-dispatch (from fastai)
  Downloading plum_dispatch-2.5.7-py3-none-any.whl.metadata (7.5 kB)
Requirement already satisfied: cloudpickle in /usr/local/lib/python3.11/dist-packages (from fastai) (3.1.1)
Requirement already satisfied: torch<2.8,>=1.10 in /usr/local/lib/python3.11/dist-packages (from fastai) (2.6.0+cu124)
Requirement already satisfied: graphviz in /usr/local/lib/python3.11/dist-packages (from fastbook) (0.21)
Requirement already satisfied: transformers in /usr/local/lib/python3.11/dist-packages (from fastbook) (4.53.2)
Requirement already satisfied: datasets in /usr/local/lib/python3.11/dist-packages (from fastbook) (2.14.4)
Requirement already satisfied: ipywidgets<8 in /usr/local/lib/python3.11/dist-packages (from fastbook) (7.7.1)
Requirement already satisfied: sentencepiece in /usr/local/lib/python3.11/dist-packages (from fastbook) (0.2.0)
Requirement already satisfied: click>=8.1.8 in /usr/local/lib/python3.11/dist-packages (from ddgs) (8.2.1)
Collecting primp>=0.15.0 (from ddgs)
  Downloading primp-0.15.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (13 kB)
Requirement already satisfied: lxml>=5.3.0 in /usr/local/lib/python3.11/dist-packages (from ddgs) (5.4.0)
Requirement already satisfied: ipykernel>=4.5.1 in /usr/local/lib/python3.11/dist-packages (from ipywidgets<8->fastbook) (6.17.1)
Requirement already satisfied: ipython-genutils~=0.2.0 in /usr/local/lib/python3.11/dist-packages (from ipywidgets<8->fastbook) (0.2.0)
Requirement already satisfied: traitlets>=4.3.1 in /usr/local/lib/python3.11/dist-packages (from ipywidgets<8->fastbook) (5.7.1)
Requirement already satisfied: widgetsnbextension~=3.6.0 in /usr/local/lib/python3.11/dist-packages (from ipywidgets<8->fastbook) (3.6.10)
Requirement already satisfied: ipython>=4.0.0 in /usr/local/lib/python3.11/dist-packages (from ipywidgets<8->fastbook) (7.34.0)
Requirement already satisfied: jupyterlab-widgets>=1.0.0 in /usr/local/lib/python3.11/dist-packages (from ipywidgets<8->fastbook) (3.0.15)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (1.0.13)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (2.0.11)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (3.0.10)
Requirement already satisfied: thinc<8.4.0,>=8.3.4 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (8.3.6)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (1.1.3)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (2.5.1)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (2.0.10)
Requirement already satisfied: weasel<0.5.0,>=0.1.0 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (0.4.1)
Requirement already satisfied: typer<1.0.0,>=0.3.0 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (0.16.0)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (4.67.1)
Requirement already satisfied: numpy>=1.19.0 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (2.0.2)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (2.11.7)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (3.1.6)
Requirement already satisfied: setuptools in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (75.2.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.11/dist-packages (from spacy<4->fastai) (3.5.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->fastai) (3.4.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->fastai) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->fastai) (2.4.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->fastai) (2025.7.14)
Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from torch<2.8,>=1.10->fastai) (3.18.0)
Requirement already satisfied: typing-extensions>=4.10.0 in /usr/local/lib/python3.11/dist-packages (from torch<2.8,>=1.10->fastai) (4.14.1)
Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch<2.8,>=1.10->fastai) (3.5)
Requirement already satisfied: fsspec in /usr/local/lib/python3.11/dist-packages (from torch<2.8,>=1.10->fastai) (2025.3.2)
Collecting nvidia-cuda-nvrtc-cu12==12.4.127 (from torch<2.8,>=1.10->fastai)
  Downloading nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-runtime-cu12==12.4.127 (from torch<2.8,>=1.10->fastai)
  Downloading nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-cupti-cu12==12.4.127 (from torch<2.8,>=1.10->fastai)
  Downloading nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cudnn-cu12==9.1.0.70 (from torch<2.8,>=1.10->fastai)
  Downloading nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cublas-cu12==12.4.5.8 (from torch<2.8,>=1.10->fastai)
  Downloading nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cufft-cu12==11.2.1.3 (from torch<2.8,>=1.10->fastai)
  Downloading nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-curand-cu12==10.3.5.147 (from torch<2.8,>=1.10->fastai)
  Downloading nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cusolver-cu12==11.6.1.9 (from torch<2.8,>=1.10->fastai)
  Downloading nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cusparse-cu12==12.3.1.170 (from torch<2.8,>=1.10->fastai)
  Downloading nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Requirement already satisfied: nvidia-cusparselt-cu12==0.6.2 in /usr/local/lib/python3.11/dist-packages (from torch<2.8,>=1.10->fastai) (0.6.2)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /usr/local/lib/python3.11/dist-packages (from torch<2.8,>=1.10->fastai) (2.21.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<2.8,>=1.10->fastai) (12.4.127)
Collecting nvidia-nvjitlink-cu12==12.4.127 (from torch<2.8,>=1.10->fastai)
  Downloading nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Requirement already satisfied: triton==3.2.0 in /usr/local/lib/python3.11/dist-packages (from torch<2.8,>=1.10->fastai) (3.2.0)
Requirement already satisfied: sympy==1.13.1 in /usr/local/lib/python3.11/dist-packages (from torch<2.8,>=1.10->fastai) (1.13.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy==1.13.1->torch<2.8,>=1.10->fastai) (1.3.0)
Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.11/dist-packages (from datasets->fastbook) (18.1.0)
Requirement already satisfied: dill<0.3.8,>=0.3.0 in /usr/local/lib/python3.11/dist-packages (from datasets->fastbook) (0.3.7)
Requirement already satisfied: xxhash in /usr/local/lib/python3.11/dist-packages (from datasets->fastbook) (3.5.0)
Requirement already satisfied: multiprocess in /usr/local/lib/python3.11/dist-packages (from datasets->fastbook) (0.70.15)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.11/dist-packages (from datasets->fastbook) (3.11.15)
Requirement already satisfied: huggingface-hub<1.0.0,>=0.14.0 in /usr/local/lib/python3.11/dist-packages (from datasets->fastbook) (0.33.4)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.11/dist-packages (from matplotlib->fastai) (1.3.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.11/dist-packages (from matplotlib->fastai) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.11/dist-packages (from matplotlib->fastai) (4.58.5)
Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.11/dist-packages (from matplotlib->fastai) (1.4.8)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.11/dist-packages (from matplotlib->fastai) (3.2.3)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.11/dist-packages (from matplotlib->fastai) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.11/dist-packages (from pandas->fastai) (2025.2)
Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.11/dist-packages (from pandas->fastai) (2025.2)
Collecting beartype>=0.16.2 (from plum-dispatch->fastai)
  Downloading beartype-0.21.0-py3-none-any.whl.metadata (33 kB)
Requirement already satisfied: rich>=10.0 in /usr/local/lib/python3.11/dist-packages (from plum-dispatch->fastai) (13.9.4)
Requirement already satisfied: joblib>=1.2.0 in /usr/local/lib/python3.11/dist-packages (from scikit-learn->fastai) (1.5.1)
Requirement already satisfied: threadpoolctl>=3.1.0 in /usr/local/lib/python3.11/dist-packages (from scikit-learn->fastai) (3.6.0)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.11/dist-packages (from transformers->fastbook) (2024.11.6)
Requirement already satisfied: tokenizers<0.22,>=0.21 in /usr/local/lib/python3.11/dist-packages (from transformers->fastbook) (0.21.2)
Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.11/dist-packages (from transformers->fastbook) (0.5.3)
Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets->fastbook) (2.6.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets->fastbook) (1.4.0)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets->fastbook) (25.3.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets->fastbook) (1.7.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets->fastbook) (6.6.3)
Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets->fastbook) (0.3.2)
Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets->fastbook) (1.20.1)
Requirement already satisfied: hf-xet<2.0.0,>=1.1.2 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub<1.0.0,>=0.14.0->datasets->fastbook) (1.1.5)
Requirement already satisfied: debugpy>=1.0 in /usr/local/lib/python3.11/dist-packages (from ipykernel>=4.5.1->ipywidgets<8->fastbook) (1.8.0)
Requirement already satisfied: jupyter-client>=6.1.12 in /usr/local/lib/python3.11/dist-packages (from ipykernel>=4.5.1->ipywidgets<8->fastbook) (6.1.12)
Requirement already satisfied: matplotlib-inline>=0.1 in /usr/local/lib/python3.11/dist-packages (from ipykernel>=4.5.1->ipywidgets<8->fastbook) (0.1.7)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.11/dist-packages (from ipykernel>=4.5.1->ipywidgets<8->fastbook) (1.6.0)
Requirement already satisfied: psutil in /usr/local/lib/python3.11/dist-packages (from ipykernel>=4.5.1->ipywidgets<8->fastbook) (5.9.5)
Requirement already satisfied: pyzmq>=17 in /usr/local/lib/python3.11/dist-packages (from ipykernel>=4.5.1->ipywidgets<8->fastbook) (24.0.1)
Requirement already satisfied: tornado>=6.1 in /usr/local/lib/python3.11/dist-packages (from ipykernel>=4.5.1->ipywidgets<8->fastbook) (6.4.2)
Collecting jedi>=0.16 (from ipython>=4.0.0->ipywidgets<8->fastbook)
  Downloading jedi-0.19.2-py2.py3-none-any.whl.metadata (22 kB)
Requirement already satisfied: decorator in /usr/local/lib/python3.11/dist-packages (from ipython>=4.0.0->ipywidgets<8->fastbook) (4.4.2)
Requirement already satisfied: pickleshare in /usr/local/lib/python3.11/dist-packages (from ipython>=4.0.0->ipywidgets<8->fastbook) (0.7.5)
Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /usr/local/lib/python3.11/dist-packages (from ipython>=4.0.0->ipywidgets<8->fastbook) (3.0.51)
Requirement already satisfied: pygments in /usr/local/lib/python3.11/dist-packages (from ipython>=4.0.0->ipywidgets<8->fastbook) (2.19.2)
Requirement already satisfied: backcall in /usr/local/lib/python3.11/dist-packages (from ipython>=4.0.0->ipywidgets<8->fastbook) (0.2.0)
Requirement already satisfied: pexpect>4.3 in /usr/local/lib/python3.11/dist-packages (from ipython>=4.0.0->ipywidgets<8->fastbook) (4.9.0)
Requirement already satisfied: language-data>=1.2 in /usr/local/lib/python3.11/dist-packages (from langcodes<4.0.0,>=3.2.0->spacy<4->fastai) (1.3.0)
Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.11/dist-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy<4->fastai) (0.7.0)
Requirement already satisfied: pydantic-core==2.33.2 in /usr/local/lib/python3.11/dist-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy<4->fastai) (2.33.2)
Requirement already satisfied: typing-inspection>=0.4.0 in /usr/local/lib/python3.11/dist-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy<4->fastai) (0.4.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.11/dist-packages (from python-dateutil>=2.7->matplotlib->fastai) (1.17.0)
Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.11/dist-packages (from rich>=10.0->plum-dispatch->fastai) (3.0.0)
Requirement already satisfied: blis<1.4.0,>=1.3.0 in /usr/local/lib/python3.11/dist-packages (from thinc<8.4.0,>=8.3.4->spacy<4->fastai) (1.3.0)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /usr/local/lib/python3.11/dist-packages (from thinc<8.4.0,>=8.3.4->spacy<4->fastai) (0.1.5)
Requirement already satisfied: shellingham>=1.3.0 in /usr/local/lib/python3.11/dist-packages (from typer<1.0.0,>=0.3.0->spacy<4->fastai) (1.5.4)
Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in /usr/local/lib/python3.11/dist-packages (from weasel<0.5.0,>=0.1.0->spacy<4->fastai) (0.21.1)
Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in /usr/local/lib/python3.11/dist-packages (from weasel<0.5.0,>=0.1.0->spacy<4->fastai) (7.3.0.post1)
Requirement already satisfied: notebook>=4.4.1 in /usr/local/lib/python3.11/dist-packages (from widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (6.5.7)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->spacy<4->fastai) (3.0.2)
Requirement already satisfied: parso<0.9.0,>=0.8.4 in /usr/local/lib/python3.11/dist-packages (from jedi>=0.16->ipython>=4.0.0->ipywidgets<8->fastbook) (0.8.4)
Requirement already satisfied: jupyter-core>=4.6.0 in /usr/local/lib/python3.11/dist-packages (from jupyter-client>=6.1.12->ipykernel>=4.5.1->ipywidgets<8->fastbook) (5.8.1)
Requirement already satisfied: marisa-trie>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from language-data>=1.2->langcodes<4.0.0,>=3.2.0->spacy<4->fastai) (1.2.1)
Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.11/dist-packages (from markdown-it-py>=2.2.0->rich>=10.0->plum-dispatch->fastai) (0.1.2)
Requirement already satisfied: argon2-cffi in /usr/local/lib/python3.11/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (25.1.0)
Requirement already satisfied: nbformat in /usr/local/lib/python3.11/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (5.10.4)
Requirement already satisfied: nbconvert>=5 in /usr/local/lib/python3.11/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (7.16.6)
Requirement already satisfied: Send2Trash>=1.8.0 in /usr/local/lib/python3.11/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (1.8.3)
Requirement already satisfied: terminado>=0.8.3 in /usr/local/lib/python3.11/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (0.18.1)
Requirement already satisfied: prometheus-client in /usr/local/lib/python3.11/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (0.22.1)
Requirement already satisfied: nbclassic>=0.4.7 in /usr/local/lib/python3.11/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (1.3.1)
Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.11/dist-packages (from pexpect>4.3->ipython>=4.0.0->ipywidgets<8->fastbook) (0.7.0)
Requirement already satisfied: wcwidth in /usr/local/lib/python3.11/dist-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=4.0.0->ipywidgets<8->fastbook) (0.2.13)
Requirement already satisfied: wrapt in /usr/local/lib/python3.11/dist-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.1.0->spacy<4->fastai) (1.17.2)
Requirement already satisfied: platformdirs>=2.5 in /usr/local/lib/python3.11/dist-packages (from jupyter-core>=4.6.0->jupyter-client>=6.1.12->ipykernel>=4.5.1->ipywidgets<8->fastbook) (4.3.8)
Requirement already satisfied: notebook-shim>=0.2.3 in /usr/local/lib/python3.11/dist-packages (from nbclassic>=0.4.7->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (0.2.4)
Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.11/dist-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (4.13.4)
Requirement already satisfied: bleach!=5.0.0 in /usr/local/lib/python3.11/dist-packages (from bleach[css]!=5.0.0->nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (6.2.0)
Requirement already satisfied: defusedxml in /usr/local/lib/python3.11/dist-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (0.7.1)
Requirement already satisfied: jupyterlab-pygments in /usr/local/lib/python3.11/dist-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (0.3.0)
Requirement already satisfied: mistune<4,>=2.0.3 in /usr/local/lib/python3.11/dist-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (3.1.3)
Requirement already satisfied: nbclient>=0.5.0 in /usr/local/lib/python3.11/dist-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (0.10.2)
Requirement already satisfied: pandocfilters>=1.4.1 in /usr/local/lib/python3.11/dist-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (1.5.1)
Requirement already satisfied: fastjsonschema>=2.15 in /usr/local/lib/python3.11/dist-packages (from nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (2.21.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.11/dist-packages (from nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (4.24.0)
Requirement already satisfied: argon2-cffi-bindings in /usr/local/lib/python3.11/dist-packages (from argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (21.2.0)
Requirement already satisfied: webencodings in /usr/local/lib/python3.11/dist-packages (from bleach!=5.0.0->bleach[css]!=5.0.0->nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (0.5.1)
Requirement already satisfied: tinycss2<1.5,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from bleach[css]!=5.0.0->nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (1.4.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.11/dist-packages (from jsonschema>=2.6->nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (2025.4.1)
Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.11/dist-packages (from jsonschema>=2.6->nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (0.36.2)
Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.11/dist-packages (from jsonschema>=2.6->nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (0.26.0)
Requirement already satisfied: jupyter-server<3,>=1.8 in /usr/local/lib/python3.11/dist-packages (from notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (1.16.0)
Requirement already satisfied: cffi>=1.0.1 in /usr/local/lib/python3.11/dist-packages (from argon2-cffi-bindings->argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (1.17.1)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.11/dist-packages (from beautifulsoup4->nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (2.7)
Requirement already satisfied: pycparser in /usr/local/lib/python3.11/dist-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (2.22)
Requirement already satisfied: anyio>=3.1.0 in /usr/local/lib/python3.11/dist-packages (from jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (4.9.0)
Requirement already satisfied: websocket-client in /usr/local/lib/python3.11/dist-packages (from jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (1.8.0)
Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.11/dist-packages (from anyio>=3.1.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8->fastbook) (1.3.1)
Downloading fastai-2.8.2-py3-none-any.whl (235 kB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m235.3/235.3 kB[0m [31m12.4 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading fastcore-1.8.5-py3-none-any.whl (79 kB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m79.3/79.3 kB[0m [31m5.0 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading fastbook-0.0.29-py3-none-any.whl (719 kB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m719.8/719.8 kB[0m [31m24.2 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading ddgs-9.3.1-py3-none-any.whl (33 kB)
Downloading fasttransform-0.0.2-py3-none-any.whl (14 kB)
Downloading primp-0.15.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m3.3/3.3 MB[0m [31m36.0 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl (363.4 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m363.4/363.4 MB[0m [31m4.3 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (13.8 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m13.8/13.8 MB[0m [31m52.5 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (24.6 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m24.6/24.6 MB[0m [31m27.9 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (883 kB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m883.7/883.7 kB[0m [31m20.1 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl (664.8 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m664.8/664.8 MB[0m [31m2.1 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl (211.5 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m211.5/211.5 MB[0m [31m6.3 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl (56.3 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m56.3/56.3 MB[0m [31m12.9 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl (127.9 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m127.9/127.9 MB[0m [31m7.5 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl (207.5 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m207.5/207.5 MB[0m [31m6.3 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (21.1 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m21.1/21.1 MB[0m [31m84.1 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading plum_dispatch-2.5.7-py3-none-any.whl (42 kB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m42.6/42.6 kB[0m [31m2.8 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading beartype-0.21.0-py3-none-any.whl (1.2 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.2/1.2 MB[0m [31m54.3 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading jedi-0.19.2-py2.py3-none-any.whl (1.6 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.6/1.6 MB[0m [31m63.4 MB/s[0m eta [36m0:00:00[0m
[?25hInstalling collected packages: primp, nvidia-nvjitlink-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, jedi, fastcore, beartype, nvidia-cusparse-cu12, nvidia-cudnn-cu12, ddgs, plum-dispatch, nvidia-cusolver-cu12, fasttransform, fastai, fastbook
  Attempting uninstall: nvidia-nvjitlink-cu12
    Found existing installation: nvidia-nvjitlink-cu12 12.5.82
    Uninstalling nvidia-nvjitlink-cu12-12.5.82:
      Successfully uninstalled nvidia-nvjitlink-cu12-12.5.82
  Attempting uninstall: nvidia-curand-cu12
    Found existing installation: nvidia-curand-cu12 10.3.6.82
    Uninstalling nvidia-curand-cu12-10.3.6.82:
      Successfully uninstalled nvidia-curand-cu12-10.3.6.82
  Attempting uninstall: nvidia-cufft-cu12
    Found existing installation: nvidia-cufft-cu12 11.2.3.61
    Uninstalling nvidia-cufft-cu12-11.2.3.61:
      Successfully uninstalled nvidia-cufft-cu12-11.2.3.61
  Attempting uninstall: nvidia-cuda-runtime-cu12
    Found existing installation: nvidia-cuda-runtime-cu12 12.5.82
    Uninstalling nvidia-cuda-runtime-cu12-12.5.82:
      Successfully uninstalled nvidia-cuda-runtime-cu12-12.5.82
  Attempting uninstall: nvidia-cuda-nvrtc-cu12
    Found existing installation: nvidia-cuda-nvrtc-cu12 12.5.82
    Uninstalling nvidia-cuda-nvrtc-cu12-12.5.82:
      Successfully uninstalled nvidia-cuda-nvrtc-cu12-12.5.82
  Attempting uninstall: nvidia-cuda-cupti-cu12
    Found existing installation: nvidia-cuda-cupti-cu12 12.5.82
    Uninstalling nvidia-cuda-cupti-cu12-12.5.82:
      Successfully uninstalled nvidia-cuda-cupti-cu12-12.5.82
  Attempting uninstall: nvidia-cublas-cu12
    Found existing installation: nvidia-cublas-cu12 12.5.3.2
    Uninstalling nvidia-cublas-cu12-12.5.3.2:
      Successfully uninstalled nvidia-cublas-cu12-12.5.3.2
  Attempting uninstall: fastcore
    Found existing installation: fastcore 1.7.29
    Uninstalling fastcore-1.7.29:
      Successfully uninstalled fastcore-1.7.29
  Attempting uninstall: nvidia-cusparse-cu12
    Found existing installation: nvidia-cusparse-cu12 12.5.1.3
    Uninstalling nvidia-cusparse-cu12-12.5.1.3:
      Successfully uninstalled nvidia-cusparse-cu12-12.5.1.3
  Attempting uninstall: nvidia-cudnn-cu12
    Found existing installation: nvidia-cudnn-cu12 9.3.0.75
    Uninstalling nvidia-cudnn-cu12-9.3.0.75:
      Successfully uninstalled nvidia-cudnn-cu12-9.3.0.75
  Attempting uninstall: nvidia-cusolver-cu12
    Found existing installation: nvidia-cusolver-cu12 11.6.3.83
    Uninstalling nvidia-cusolver-cu12-11.6.3.83:
      Successfully uninstalled nvidia-cusolver-cu12-11.6.3.83
  Attempting uninstall: fastai
    Found existing installation: fastai 2.7.19
    Uninstalling fastai-2.7.19:
      Successfully uninstalled fastai-2.7.19
Successfully installed beartype-0.21.0 ddgs-9.3.1 fastai-2.8.2 fastbook-0.0.29 fastcore-1.8.5 fasttransform-0.0.2 jedi-0.19.2 nvidia-cublas-cu12-12.4.5.8 nvidia-cuda-cupti-cu12-12.4.127 nvidia-cuda-nvrtc-cu12-12.4.127 nvidia-cuda-runtime-cu12-12.4.127 nvidia-cudnn-cu12-9.1.0.70 nvidia-cufft-cu12-11.2.1.3 nvidia-curand-cu12-10.3.5.147 nvidia-cusolver-cu12-11.6.1.9 nvidia-cusparse-cu12-12.3.1.170 nvidia-nvjitlink-cu12-12.4.127 plum-dispatch-2.5.7 primp-0.15.0
Mounted at /content/gdrive

from fastbook import *
from fastai.vision.widgets import *
from ddgs import DDGS
import time, random
from pathlib import Path

# 1. Collect data
animals = ["cat", "dog"]
path = Path('animals')  # Path to store images

if not path.exists():
    path.mkdir()

for animal in animals:
    dest = (path/animal)
    dest.mkdir(exist_ok=True)
    with DDGS() as ddg:
        results = [r for r in ddg.images(f'{animal}', max_results=100)]  # Search for cat or dog images
        # Extract content URLs from the result dictionaries
        urls = [r.get('image') or r.get('contentUrl') for r in results if r.get('image') or r.get('contentUrl')]
        # Random delay to avoid spamming the server
        time.sleep(random.uniform(1, 3))
        # Download the images
        download_images(dest, urls=urls)

# List results
fns = get_image_files(path)
print(len(fns))

Now that the process of collecting data is over, we need to check if any are corrupted. By corrupted, we usually refer to files that return an error when trying to open them. An image or file is corrupted when: * Downloaded incompletly * Subject to network tranmission errors * Storage or memory issues * File-system or server-side problems and if that’s the case clean them.

To check for integrity, we use the verify_images function. This will return a collection object that has a map method which calls the passed function onto each element of the collection. In this case it’s the unlink function that will handle the cleaning of corrupted files.

failed = verify_images(fns)
print(failed)
failed.map(Path.unlink);

/usr/local/lib/python3.11/dist-packages/PIL/Image.py:3570: UserWarning: image file could not be identified because AVIF support not installed
  warnings.warn(message)
/usr/local/lib/python3.11/dist-packages/PIL/Image.py:3570: UserWarning: image file could not be identified because AVIF support not installed
  warnings.warn(message)


[Path('animals/cat/d97e5771-40d9-4a1e-b1a3-849a32bca85a.jpg'), Path('animals/dog/743b109f-0222-4cc2-b1aa-fa95f3811185.jpg'), Path('animals/dog/fddc2b23-6712-4803-838d-81729f8efef5.jpg'), Path('animals/dog/3c41c6ee-858c-4b6d-9660-fbacab718d1c.jpg'), Path('animals/dog/c517196a-abde-49a6-b28c-35c4980dd228.jpg'), Path('animals/dog/f5b15c94-0dc4-4c75-b7f7-7cb7c3cc8854.jpg'), Path('animals/dog/97123c37-6e78-4ee9-9e8c-f1b85314e046.jpg'), Path('animals/dog/684ef9f1-9b6e-4154-bf3f-a077657ed265.jpg'), Path('animals/dog/d0fb283f-eee8-40f2-a1da-6195368eb85e.jpg')]

Now that we have our dataset, it’s time to split it into train, validation and test sets. * Training Set: Used to teach the model.

Validation Set: Used during development to tune model settings (hyperparameters) and check generalization.
Test Set: Used only at the end to fairly evaluate final model performance on unseen data.

Without separate sets, the model (or developers) might overfit—performing well on known data but poorly on new data. The test set ensures honest, unbiased evaluation and helps prevent self-deception or vendor manipulation.

Keep in mind that data is usually biased, and if that’s not taken into consideration, your model will end up equally biased.

To achieve that we use a DataLoader on each set and a thin super class DataLoaders to combine them all.

# 3. Define the DataBlock
catVsDog = DataBlock(
    blocks=(ImageBlock, CategoryBlock),                 # Image = input, Category = output
    get_items=get_image_files,                          # Collect all image file paths
    splitter=RandomSplitter(valid_pct=0.2, seed=42),    # 80/20 train/val split
    get_y=parent_label,                                 # Label = name of parent folder (e.g. 'cat' or 'dog')
    item_tfms=RandomResizedCrop(224, min_scale=0.5),    # Resize and crop images
    batch_tfms=aug_transforms(mult=2)                   # Augmentations like flip, rotate, etc.
)

# 4. Create DataLoaders
try:
    dls = catVsDog.dataloaders(path)
    print("DataLoaders created successfully!")

    # Show a sample batch to verify everything works
    dls.show_batch(max_n=8)

    # 5. Create learner with resnet18 and track error rate
    learn = vision_learner(dls, resnet18, metrics=error_rate)

    # 6. Train the model for 4 epochs with fine-tuning
    learn.fine_tune(4)

except Exception as e:
    print(f"Error creating DataLoaders: {e}")
    print("This might be due to remaining corrupted images. Let's do a more thorough cleanup...")

DataLoaders created successfully!


Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /root/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth
100%|██████████| 44.7M/44.7M [00:00<00:00, 107MB/s]

epoch	train_loss	valid_loss	error_rate	time
0	1.476215	0.458743	0.171429	00:40

epoch	train_loss	valid_loss	error_rate	time
0	0.419707	0.177242	0.057143	00:58
1	0.326388	0.171967	0.028571	00:54
2	0.235696	0.221325	0.057143	00:58
3	0.188272	0.234519	0.057143	00:56

The train and validation losses seem alright. But we still need to see where exactly our errors are occuring.

interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix()

interp.plot_top_losses(5, nrows=1)

There is clearly an image where I have both animals but labelled as cat so I choose <Delete>. As for the small doggo being mistaken for a cat, work hazard I suppose.

cleaner = ImageClassifierCleaner(learn)
cleaner

VBox(children=(Dropdown(options=('cat', 'dog'), value='cat'), Dropdown(options=('Train', 'Valid'), value='Trai…

for idx in cleaner.delete(): cleaner.fns[idx].unlink()
for idx,cat in cleaner.change(): shutil.move(str(cleaner.fns[idx]), path/cat)

Let’s re-train for another epoch or two now that the data is cleaned.

# We keep the same transforms etc.
dls = catVsDog.dataloaders(path)
print("DataLoaders created successfully!")

# Re-create learner with resnet18 and track error rate
learn = vision_learner(dls, resnet18, metrics=error_rate)

# Train the model for 4 epochs with fine-tuning
learn.fine_tune(4)

DataLoaders created successfully!

epoch	train_loss	valid_loss	error_rate	time
0	1.132787	0.483606	0.212121	00:42

epoch	train_loss	valid_loss	error_rate	time
0	0.372304	0.206328	0.030303	00:58
1	0.317983	0.132354	0.030303	01:00
2	0.241134	0.078701	0.030303	00:58
3	0.188229	0.051700	0.030303	01:04

2 Turning the Model into a Web Application

Once the model fine tuned, it’s time to save it so it can be used in production.

learn.export("CatVsDog.pkl")

# Make sure the model is correctly saved
path = Path()
path.ls(file_exts='.pkl')

(#1) [Path('CatVsDog.pkl')]

Time to use this inference learned in a web application. For that I opted for gradio for fast showcasing. In the book, however they used voila along with jupyter notebook for deployment.

!pip install gradio

import gradio as gr
from fastai.vision.all import *

# Load your model
learn = load_learner('export.pkl')  # Your Fastai model

# Define prediction function
def predict(img):
    _, _, probs = learn.predict(img)
    return {learn.dls.vocab[i]: float(probs[i]) for i in range(len(probs))}

# Build interface
interface = gr.Interface(fn=predict, inputs=gr.Image(type="pil"), outputs=gr.Label())
interface.launch()

3 Notes on Development & Deployment

While working on this first post, I encountered and resolved several challenges worth documenting.

Initially, all development—including model training and testing—was done in a Google Colab notebook. I wanted to preserve the cell outputs from that session to accurately reflect the training process in the final blog post. This meant converting the Colab notebook into a Quarto .qmd file without re-executing the code during rendering, so outputs would remain intact.

To prevent code re-execution, I added the following to both the YAML header of the .qmd file and the global Quarto configuration:

title: Title
jupyter: python3
execute:
  freeze: true

However, using quarto convert my-notebook.ipynb didn’t preserve the output cells as expected. A reliable workaround was to first convert the notebook to Markdown:

jupyter nbconvert my-notebook.ipynb --to markdown --output converted-notebook.qmd

Then, I renamed the file to .qmd and manually added the necessary YAML header.

Another issue arose when I attempted to run the exported model on my local Windows machine. Since the model was trained in a Unix environment, it used PosixPath internally (via pathlib), which led to this error on Windows:

NotImplementedError: cannot instantiate 'PosixPath' on your system

To resolve this, I patched PosixPath to WindowsPath during model loading:

import platform
from pathlib import Path
if platform.system() == 'Windows':
    import pathlib
    pathlib.PosixPath = pathlib.WindowsPath

Lastly, for consistent predictions, ensure your local environment mirrors the dependency versions used during model training—particularly for fastai, torch, and torchvision.