In the Shackled City Adventure Path - in Dungeon magazine and published by Paizo as a book - there was a 3.5e adventure about the Flood Festival in Cauldron (Greyhawk) that had some fun contests and a ball. Fan authors added more In Paizo’s discussions of “SCAP”.
In my 3.5e Greyhawk campaign, I had a Midsummer Festival at the Keep on the Borderlands with archery, wrestling, and swimming contests (you can figure out rules).
And I had a baking contest and singing contest to suit my players’ interests and their characters abilities. Simple skill checks for the PC and some NPC’s.
Prizes were related to the contests, like a bow or a gig singing at the inn, but it was mostly for fun.
I didn’t think about Special Olympics, but definitely go for it. Your own creativity on contests will be great!