I deliver training classes and workshops at the company I work for. One of the most popular set of workshops is a series on the PyTorch neural code library. The fundamental objects in PyTorch are tensors. A tensor is a variable dimension sized array/matrix that can be processed on a GPU.
Python 3 Quick Reference Card
Learning about PyTorch tensors isn’t difficult, but it’s difficult — an individual topic is relatively simple but there are probably about 100 significant topics.
Pytorch Summary sheet. Some variations: torch.addcmul(tensor, value=1, tensor1, tensor2): outi = tensori + value × tensor1i × tensor2itorch.addcdiv(tensor, value=1, tensor1, tensor2): outi = tensori + value × tensor1i / tensor2iOther element wise math function. PYTHON CHEAT SHEET Python is a most popular general-purpose, high-level programming language which was created in 1991 by Guido van Rossum and developed by Python Software Foundation to concentrate on the readability of code with its extensive use of white space.
So, when I’m delivering training, I start with tensor basics. The challenge isn’t what to discuss, it’s what to skip.
Here’s my current version of a demo program that I consider a good starting point.
There are dozens of ways to do anything in PyTorch. This fact is one of the biggest weaknesses of the library’s design. I think every PyTorch program should document the versions of PyTorch and Python used, and set the numpy and torch random generator seed.
PyTorch has nine basic data types that can be used when creating a tensor, but the two most common are type float32 (alias float, alias FloatTensor alias cuda.FloatTensor) and int64 (alias long, alias . . . ) so my cheat sheet shows just two of the nine types. One of the weird quirks of PyTorch is that the value of a tensor that holds a single numeric value can be fetched using the item() method. You rarely create an empty tensor but you can do so in many ways, for example,
t = T.zeros((2,3), dtype=T.float32).to(device) # 2×3 all 0.0s
When using PyTorch to create deep neural models, you almost always have to reshape tensors. This is always tricky and in my experience, one of the most common sources of error during development. So, I give examples of reshape(), view(), squeeze(), and flatten().
Pytorch Cheat Sheet 2020
Converting a PyTorch tensor to a numpy array or a Python list isn’t needed in all PyTorch programs, but such conversions happen often enough that I give a couple of examples.
By far the trickiest aspect of working with PyTorch tensors is using the dim parameter that exists in most functions that work on tensors. Common functions used by most programs are softmax() and max().
Pytorch Lightning Cheat Sheet
Old science fiction movie posters sometimes exaggerated what viewers would actually see. I guess you could say these posters were sort of cinematic cheat sheets. Here are three that feature robots. Left: “Forbidden Planet” (1956) – one of the best science fiction movies ever made. Center: “The Colossus of New York” (1958) – not one of the best science fiction movies ever made. Right: “The Day the Earth Stood Still” (1951) – Klaatu barada nikto.