Sidney Freedman of the Precast/Prestressed Concrete Institute says this is caused by differences in aggregate orientation on the two faces. If aggregates aren't close to being cubical in shape, consolidation causes them to settle with the flat, long portion horizontal to the bottom of the mold. These flat faces then show when they're exposed. However, on the return (the channel section at an angle to the down face), the points of the aggregate show when they're exposed.
For sections with deep returns, Freedman recommends using two-stage or sequential precasting. This involves casting the section in separate pieces that are joined with reinforcing steel. This allows all faces to be cast face down with the same aggregate orientation.
For more information on precast architectural concrete, we suggest you purchase a copy of PCI's Collection of Ideas on the Production of Architectural Precast Concrete.
You can order it from the Precast/Prestessed Concrete Institute, 175 W. Jackson Blvd., Chicago, IL 60604. The cost is $15 to PCI members and $30 to nonmembers.