In CSE 158 - Recommender Systems, as a part of my final project, I worked within a team to develop a computer vision and collaborative filter shoe recommender based around an Amazon co-purchase dataset.

The model was trained to predict co-purchase behavior (e.x. given anonymized user X bought Y, how likely they are to buy Z). Furthermore, it is also evident from fashion trends that the visual appearance of an item will also influence whether or not the item is co-purchased, similar to a user’s previous shopping trends.

Non-Visual Model

A non-visual model was constructed using a Factorization Machine of the dataset features.

Visual Model

Our visual model was trained using additional image embedding features, obtained by extract last-layer representations of passing the images through AlexNet. Then, the Mahalobis distance of image similarities (k=1, 10. 100) were learned and passed into a Binary Classifier using a modified sigmoidal activation function, trained using PyTorch Lightning.