Why the hamster result is a surprise
The Mel1b receptor was long assumed to be the main route through which melatonin signals seasonal and circadian information. The hamster study tested this directly by using animals that lacked a functional Mel1b receptor. Those animals still showed seasonal reproductive responses and circadian responses to melatonin, which indicates that the Mel1b receptor is not necessary for these particular responses in this species [1].
This does not mean the receptor does nothing. It means that, at least for the reproductive and circadian responses measured in Siberian hamsters, another pathway can carry the melatonin signal. The finding is specific to those responses and that species; it does not rule out a role for Mel1b in other tissues or other aspects of melatonin biology.
What the mouse evidence actually shows
The mouse study is not a test of seasonal reproduction or of the hamster's circadian responses. It examines how melatonin protects hippocampal neurons from ischemic injury. In that setting, melatonin and a melatonin receptor agonist suppressed NMDAR currents, and a melatonin receptor antagonist blocked the effect, which suggests that melatonin receptors—likely including Mel1b—are involved in this neuroprotective response [2].
That is a different physiological question from the hamster one. A receptor can be required for one response and dispensable for another, so the mouse neuroprotection result and the hamster reproductive result are not in direct conflict. They simply do not measure the same outcome.
Species and response type change the answer
The broader literature reinforces that the role of Mel1b depends on what is being measured. In chicks, Mel1b receptors in the liver show a circadian rhythm and can induce clock gene expression when activated, which is consistent with a role in circadian regulation in that tissue and species [4]. In humans, a common variant in the MTNR1B gene (which encodes Mel1b) is associated with disturbed circadian phenotypes and altered melatonin secretion, and with type 2 diabetes risk [3].
These findings do not contradict the hamster result. They show that Mel1b can matter for circadian and metabolic traits in some contexts while being unnecessary for the specific reproductive and circadian responses tested in Siberian hamsters. The honest summary is that hamster and mouse findings do not agree or disagree in any simple way, because they address different responses in different species.
Sources used in this answer
Nature's knockout: the Mel1b receptor is not necessary for reproductive and circadian responses to melatonin in Siberian hamsters.
Siberian hamsters lacking a functional Mel1b receptor still show seasonal reproductive and circadian responses to melatonin, indicating the receptor is not required for these responses.
Melatonin-Induced Postconditioning Suppresses NMDA Receptor through Opening of the Mitochondrial Permeability Transition Pore via Melatonin Receptor in Mouse Neurons
In mouse hippocampal neurons, melatonin and a melatonin receptor agonist suppressed NMDAR currents, and a melatonin receptor antagonist blocked this effect, suggesting melatonin receptors mediate this neuroprotective response.
The melatonin receptor 1B gene links circadian rhythms and type 2 diabetes mellitus: an evolutionary story
The MTNR1B rs10830963 G allele is associated with disturbed circadian phenotypes, altered melatonin secretion, and type 2 diabetes risk, linking the Mel1b gene to circadian and metabolic traits in humans.
Circadian rhythm of melatonin receptors under monochromatic light and their regulatory effect on clock genes in the chick liver
In chick liver, Mel1b receptors show a circadian rhythm and their activation induces expression of several clock genes, consistent with a role in circadian regulation in that tissue.
… Receptors Mediate Somatic, Behavioral, and Reproductive Neuroendocrine Responses to Photoperiod and Melatonin in Siberian Hamsters (Phodopus sungorus)
Siberian hamsters show somatic, behavioral, and reproductive neuroendocrine responses to photoperiod and melatonin, but the abstract does not explicitly address the Mel1b receptor.