While athletes would still participate, Ms Pelosi said heads of state should not attend.
Ms Pelosi said the US could not proceed as if nothing was wrong with the Olympics being held in China.
She is the latest American politician to call for a boycott over the treatment of Uyghur Muslims.
China is believed to have detained more than a million Uyghurs and other Muslims in Xinjiang, a large region in north-western China.
Most of those detained in Xinjiang are sent to “re-education” facilities – prison-like camps where they are held for indeterminate periods of time without charge.
Ms Pelosi, a Democrat, told a bipartisan congressional hearing on the issue that heads of state around the world should boycott the Winter Olympics, which are to take place in February.
“Let’s not honour the Chinese government by having heads of state go to China,” she said.
“For heads of state to go to China in light of a genocide that is ongoing – while you’re sitting there in your seat – really begs the question, what moral authority do you have to speak again about human rights any place in the world?” she said.
In response, Liu Pengyu, a spokesman for the Chinese embassy in Washington, was scathing of Ms Pelosi’s proposal.
“I wonder what makes some US politicians think they actually have the so-called ‘moral authority’? On human rights issues, they are in no position, either historically or currently, to make wanton groundless criticism against China,” Mr Liu told the Reuters news agency.
