Patrick Awuah left Ghana in the 1980s with “$50 in his pocket” to take up a scholarship in the US.
But after a career with Microsoft, he returned to found Ashesi University College, and has now won the Wise Prize, worth $500,000 (£380,000).
Mr Awuah said there was an “urgent need” to improve education in Africa.
The prize, awarded at the World Innovation Summit for Education in the Gulf state of Qatar, was in recognition of Mr Awuah’s work as founder of Ashesi University College.
The private, not-for-profit college was intended to help develop a new generation of leaders for Ghana and other African countries.
After working and studying in the US, Mr Awuah had decided to create a university in his own country to fill what he saw as a gap in institutions which had an “ethical mindset”.
Ashesi University has emphasised the importance of ethics and entrepreneurship
Ashesi opened with 30 students in 2002 and is now teaching about 800 students on a campus outside Accra.
Mr Awuah said that the university provided a “broader perspective” than simply studying for a qualification for a job.
“We started with the idea of ethical leaders and entrepreneurial leaders.”
Students were encouraged “to ponder the big questions about what kind of country we want”.
About half of the university’s intake of students are on scholarships, half are women and they come from 18 African countries. All students have to study ethics and entrepreneurship and have to perform some community service.
Mr Awuah told the BBC he wanted to see more collaboration to raise standards in African universities.
“What we’re trying to achieve is a network of high-quality institutions in Africa – and to be more intentional about teaching future leaders.
“Not just political leaders, but judges, doctors and teachers – the people who will influence other people.”
There was also a need for more investment in higher education, he said.
“If we are to lift the continent up, we need to educate great scientists and engineers.
“We need to link with other universities around the world.”
Mr Awuah wants to build a network to raise higher education standards across Africa
International education targets, such as in the millennium development goals, tended to focus on tackling the gaps in primary and secondary education in sub-Saharan Africa, rather than university level.
And there have been warnings of a lack of investment in the university sector in Africa.
In the most recent QS World University Rankings, there were only five universities from Africa in the top 500, with three in South Africa and two in Egypt.
“This is a crucial moment for Africa. Today, one out of six people on Earth lives in Africa, and this is set to rise to one in four by 2050. We urgently need to boost the education system in Africa,” said Mr Awuah.
The prize was presented at the Wise education summit in Doha by Sheikha Moza bint Nasser, chairwoman of the Qatar Foundation.
Previous winners have included Sakena Yacoobi, for her work protecting education in conflict zones in Afghanistan; Ann Cotton for supporting education for girls in Africa and Vicky Colbert, for providing education for disadvantaged communities in Colombia.