Out of 21 urban school systems, Cleveland's fourth and eighth grade students each scored second-to-last on the NAEP exam in both math and reading.
Cleveland is the only Ohio district to take part in the Trial Urban District Assessment, or TUDA, which compares their NAEP scores to those of schools in other large cities.
The Fordham Institute's policy analyst Aaron Churchill said the results show that the district has to make dramatic improvements to better prepare their students for the future.
"They have mountains to go in terms of preparing their kids ready to go to college, whether it's a two or four year college, or into a career that's meaningful and gainful," Churchill said.
Only Detroit scored lower than Cleveland in every category, while cities like Los Angeles and Washington D.C. showed gains in math and reading.
Cleveland hasn't shown any statistically significant progress since it became part of the TUDA group in 2003.
Churchill is hoping that an improvement strategy and new state law known as the Cleveland Plan will help boost the district's NAEP scores next time around.
"I wouldn't go out of my way and predict 10, 20, 30 point gains," he said. "But hopefully we'll see something significant in 2015."