Yes actually! Be sure to mark the negative side with red ink so that you keep track of which side is which

The higher the temperature, the easier it is for electrons to hop between the metals, and the higher the voltage produced. Most multimeters are sensitive enough to measure this voltage, which can be cross referenced against a chart to figure out the temperature