Here are all the places where VSCode stores stuff on Mac OS X, besides the Visual Studio Code.app itself, which is in your Applications folder:
rm -fr ~/Library/Preferences/com.microsoft.VSCode.helper.plist
rm -fr ~/Library/Preferences/com.microsoft.VSCode.plist
rm -fr ~/Library/Caches/com.microsoft.VSCode
rm -fr ~/Library/Caches/com.microsoft.VSCode.ShipIt/
rm -fr ~/Library/Application\ Support/Code/
rm -fr ~/Library/Saved\ Application\ State/com.microsoft.VSCode.savedState/
rm -fr ~/.vscode/
Update (Feb 2020): There are potentially also hidden extension directories in your home directories. To get rid of everything make sure you look for those too. They start with .vscode-.
Please run this command with care. Maybe you want to keep extension directories.
rm -rf ~/.vscode*