There will be no gigs, however, or actual instruments, with the group reserving their performances for goal celebrations.
Dubbed the "Backstreet Moyes", Rice says the trio planned to celebrate together if they scored, and they managed to do just that after Lingard's goal in last month's 2-1 win over Tottenham Hotspur.
“It was planned," Rice says in the latest episode of Uncut on the Premier League's YouTube channel.
"Jesse, when he was at Manchester United, did the celebration where he’s got the [saxophone] and he moves his legs, and then when he came in to West Ham, we do a game. It’s called "D", where the keeper pings it out, you have to take one touch and score.
"Pablo scored and started doing it and Jesse was crying with laughter, just that Pablo was doing it. So before the game we were in the changing room and Pablo was like, 'Look, if we score today, Jesse you go on the [saxophone], Dec you go on the drums and I’ll go on the guitar.'
"Then luckily enough Jesse scored and we did the celebration.”
In the video, Rice also opens up on his friendship with Chelsea midfielder Mason Mount, and praises captain Mark Noble for helping him lead the team and the example he has set for the rest of the squad.
"Nobes has been great really," he says. "I think if I didn’t actually have him here we probably wouldn’t have been as successful this season. Having him around, even though he hasn’t played as much, has still been a massive impact for everyone."