The Fifth One (mainly referred to just as The One at the time) was a legendary human male born inside the Matrix who somehow became powerful enough to manipulate nearly any aspect of the Matrix to his liking. Referred to as "The Anomaly" by the Machines, he was mysteriously exempt from the rules of the system and was literally capable of altering its structural integrity.
With this power, he managed to free the first members of the Resistance from the Power Plant that would go on to build Zion. Upon his death, the Oracle prophesied he would eventually return in some form and completely destroy the Matrix, freeing all of humanity, ending the Machine War once and for all. Since then, countless Zion operatives have searched the Matrix endlessly for him.
It wasn't until, approximately, a century later when Morpheus, captain of the Nebuchadnezzar believed he had found the One in his latest redpill, Neo. Morpheus told the story of the fifth One to Neo as an explanation of how the Resistance was formed and to also explain why he freed Neo's mind at such an older age than is normally considered as he truly believed Neo was, indeed, his successor.
Later, Neo realizes that he is indeed the new One. Upon hearing the news, Zion rejoiced as Neo began to accept his role as their savior.
The truth behind the legend
The Path of the One, also referred to as the Prime Program, would grant a randomly selected bluepill powers derived directly from The Source and would typically culminate with the One meeting the Architect. The Architect would then grant them a choice to either select 23 individuals to rebuild the about-to-be-destroyed Zion or to let humanity become extinct. Left with really no option, The One, of course, would naturally choose the former. Those 23 people who were freed from the Matrix would eventually come to believe that they were the first members of the Resistance with The One who freed them also believed to be the first.
This cycle had been repeated 5 times with the Fifth One ultimately being the last of Neo's predecessors to choose Zion new founders.