Inside-Out Planet Formation (IOPF) is a theory addressing the origin of Systems of Tightly-Packed Inner Planets (STIPs) via in-situ formation and growth of the planets. It predicts that a pebble ring is established at the pressure maximum associated with the dead zone inner boundary (DZIB) with an inner disk magnetorotational instability (MRI)-active region. Using direct N-body simulations, we study the collisional evolution of planetesimals formed from such a pebble ring, in particular examining whether a single dominant planet emerges. We consider a variety of models, including some in which the planetesimals are continuing to grow via pebble accretion. We find that the planetesimal ring undergoes oligarchic evolution, and typically turns into 2 or 3 surviving oligarchs on nearly coplanar and circular orbits, independent of the explored initial conditions or form of pebble accretion. The most massive oligarchs typically consist of about 70% of the total mass, with the building-up process typically finishing within ∼105 years. However, a relatively massive secondary planet always remains with ∼30−65% of the mass of the primary. Such secondary planets have properties that are inconsistent with the observed properties of the innermost pairs of planets in STIPs. Thus, for IOPF to be a viable theory for STIP formation, it needs to be shown how oligarchic growth of a relatively massive secondary from the initial pebble ring can be avoided. We discuss some potential additional physical processes that should be included in the modeling and explored as next steps.