Ms Rousseff, 62, who has never before held elected office, becomes the country's first woman president.
She promised to "honour the trust" Brazilians had put in her and work to eliminate poverty.
Ms Rousseff was the preferred successor of President Lula, who is leaving after two terms with record popularity.
Thousands of supporters of the governing Workers' Party took to the streets across Brazil to celebrate her victory.
The Superior Electoral Court said that with almost all the votes counted, she had won 56% against 44% for her rival, Jose Serra of the Social Democratic Party.
Although voting is compulsory in Brazil, there was a high rate of abstention at 21.5%.
The second round of voting was forced after Ms Rousseff fell short of the 50% needed in the 3 October first round, winning 47% to Mr Serra's 33%.'Fundamental promise'
In her victory speech, she said her first priority would be to lift 20 million Brazilians out of poverty.
"I reiterate my fundamental promise: the eradication of poverty," she said.
"We must not rest while there are Brazilians going hungry."
Her election as the country's first female leader, Ms Rousseff said, was a sign of the democratic progress Brazil had made.
Her priority now was to make sure that such equality of opportunity between men and women became the norm at every level, she added.
"I would like parents who have daughters to look straight in their eyes and tell them: 'Yes, a woman can.'"
Ms Rousseff, who will be sworn in on 1 January, is expected to continue the left-leaning policies of President Lula, with emphasis on government efficiency, expanding the role of the state in some sectors such as mining, and upgrading the country's decrepit infrastructure.
She will also oversee a huge expansion of Brazil's oil industry, following the discovery of major offshore fields that should make Brazil one of the world's top 10 oil exporters.
She can count on strengthened majorities for the governing coalition in both houses of Congress to help ease the task of pushing her legislative agenda.Lula effect
Ms Rousseff's victory owed much to the extraordinary popularity of the outgoing President Lula, who endorsed her as his successor from the start.
It wasn't the outright first round win President Lula had hoped for, but in the end he has managed to ensure his preferred successor, Dilma Rousseff, was elected.
Jose Serra's campaign was an uphill struggle against a president boasting approval ratings of about 80%. But even though the second round campaign was heated, with many personal attacks and corruption allegations, the candidates didn't differ much in what they had to offer to voters, nor go into great detail over their programmes.
Brazilians are trusting that Ms Rousseff will be able to build on President Lula's social and economic achievements. But they do not have a clear picture of the first woman to be elected for the highest office in the country.
She's considered a tough and efficient manager but she hasn't yet shown her political skills, especially when compared to President Lula, who's often described as a master negotiator.
Mr Lula, who has to step down after completing the maximum allowed two consecutive terms, said he would not interfere in her government.
Ms Rousseff will have "to form a government in her own image. I only hope she achieves more than I did", he said after casting his vote.
He added that he would not be attending public victory celebrations because "this is her party".
Ms Rousseff paid tribute to her mentor, saying: "I will be knocking on his door often, which, I'm sure, will always be open."
Succeeding Lula would be "difficult and challenging", she said, "but I know how to honour his legacy. I know how to consolidate and advance his work."
A former Marxist rebel who was jailed and tortured in 1970-72 for resisting military rule, Ms Rousseff trained as an economist and worked her way up through local and state governments.
She joined President Lula's cabinet as energy minister in 2003-5 and then became his chief of staff.
For Jose Serra, this is the second time he has been defeated in a presidential run-off, after losing to Mr Lula in 2002.
He has congratulated Ms Rousseff and said he hoped she would work for the good of the country.
He said: "I proudly battled the president. To those of us imagining we're defeated: We have only started the real fight."