Short answer: don’t use an Image column, but an Image URL column type. Then your formula to get
User.Photo will work just fine, and you’ll get the image displayed in cards, popup layouts etc:
Column types Image and File are for uploads only. It seems like you just cannot get anything into them with formulas — only through a file upload.