使用Pandas，Numpy解析的MNIST数据

首先是参考网站：

• http://yann.lecun.com/exdb/mnist/
• https://docs.python.org/zh-cn/3.8/library/struct.html
• https://docs.python.org/zh-cn/3.8/tutorial/inputoutput.html#tut-files
• https://blog.csdn.net/simple_the_best/article/details/75267863
• https://numpy.org/doc/stable/reference/generated/numpy.fromfile.html

然后就是具体咋写的

from typing import Tuple
import pandas as pd
import numpy as np
import sys
import struct


def read_label_in_idx1_ubyte(path: str) -> Tuple[int, int, pd.DataFrame]:
    file = open(path, mode="rb")
    magic_number, count = struct.unpack(">ii", file.read(8))
    labels = np.fromfile(file=file, dtype=np.uint8)
    labels = pd.DataFrame(labels)
    return magic_number, count, labels


def read_image_in_idx3_ubyte(path: str) -> Tuple[int, int, int, int, pd.DataFrame]:
    file = open(path, mode="rb")
    magic_number, count, rows, columns = struct.unpack(">iiii", file.read(16))
    images: np.array = np.fromfile(file=file, dtype=np.uint8)
    images = images.reshape(count, (rows*columns))
    images = pd.DataFrame(images)
    return magic_number,count,rows,columns,images


labels = read_label_in_idx1_ubyte(
    "MNIST/train-labels-idx1-ubyte/train-labels.idx1-ubyte")[-1]
image = read_image_in_idx3_ubyte(
    "MNIST/train-images-idx3-ubyte/train-images.idx3-ubyte")[-1]

说点坑，首先这个必须用二进制流打开，然后，是大端模式。

使用Pandas，Numpy解析的MNIST数据

原文：https://www.cnblogs.com/Lemon-GPU/p/15259853.html