What Happens to a Constant Current Source with No Load?

There is no such thing as a true ideal current source. The closest thing that comes to one is a particle accelerator. So you need to fabricate a Norton equivalent of your current sourcing circuitry to allow you to do your own analysis. Keep in mind that there will be a limited linear range that your circuitry will operate as a current source. The most likely limitation with a LM317 will be the 3V drop required across the device.

As for your frequency concern, 2 kHz will probably not be a problem for this semiconductor. However, switching your load impedance at any frequency will reveal problems in your understanding of the circuit and actual limitations of the circuit at that switching frequency.

An example in simpler terms. If you designed the circuit to provide 1mA of current and in all states (ON, OFF, and transition) there is a current path for this 1mA that the LM317 stays linear then the LM317 will provide a constant 1mA of current. However, if in any of the states the LM317 becomes non-linear then 1mA of current will not be maintained. I would expect that when one switched into a linear load condition at 2kHz that the LM317 will virtually instantly become linear and 1mA will flow. Now parasitic capacitances and inductances everywhere in the circuitry may alter waveforms from what you expect to see, but that's a different problem.

For stability reasons the lm317 needs a minimum load current about 5mA. Also for a real current source with 317 the output capacitor is out of the question, something that adds to instability. If you solve those two, and keep input power under 40V, 2Khz intermittent is within reason. But you WILL have current bursts at loading, and your load circuit must be able to survive them. S.M.