Jeremy Clarkson got rattled when it was put to him that he bought a farm to avoid inheritance tax.