Interpolations

Both the PredictionManager and NetworkObject have interpolation values, but with separate objectives

PredictionManager interpolation is not the same as NetworkObject interpolation.

Interpolation on the PredictionManager holds a number of states in queue as interpolation while NetworkObject interpolation handles graphical smoothing after running states.

A prediction interpolation of 0, be it client or server setting, means that states would be run soon as they are received. Most games will not use 0 interpolation as it's generally best to have at least 1 state in queue to compensate for latency. For every 1 interpolation added is like adding latency to when spectated states are run, equal to (interpolation * TickDelta).

NetworkObject interpolation determines over how many ticks the graphics are smoothed. If you were to use an interpolation of 1 for the owner of the object, then every graphical update would move over 1 tick. This is the same as interpolation on most other networking solutions or components, like the NetworkTransform.

Under the scenario your NetworkObject graphical had a set spectator interpolation of 2 and your PredictionManager an interpolation of 1, the spectated graphical object would take 2 ticks to get to it's goal after the PredictionManager ran the state 1 interpolation later than receiving it.