As for diesel fuel being a "waste" product of the refining process, well, obviously not true but, as in many things, there is a historical grain of truth which explains how it got started many years ago.

There is an optimum breakdown of distillation products from a particular crude that will provide the easiest and most inexpensive set of output products. The ability to change the breakdown to a different mix of outputs has improved immensely over the years, and it is amazing what they can do to crude oil nowadays to get more of the lighter components, such as gasoline, while reducing heavier components. Cracking was the first of the technologies which allowed lighter fractions to be increased.

History of Gasoline

Many years ago before technologies were so advanced, there was a greater demand for the lighter/gasoline fractions and there tended to be an excess of the heavier fractions coming out of distillation processes due to a high demand for gasoline and lower demand for diesel (compared to the "normal" distillation breakdown of crude), e.g., many locomotives were still using coal, rather than diesel, even after WWII, and diesel truck usage was not as great vs. gasoline.

So while I was quite young in the 1950's, I recall that diesel fuel was dirt cheap in relation to gasoline, because there was a natural excess supply of diesel from refineries and, as in most supply and demand situations, the excess supply caused the low price.

So while diesel was never really a "waste product" of the refining process, it was sort of a component most refineries wanted to reduce and was considered an undesirable output of the process.

Since then, expanded diesel fuel applications have increased demand and refining processes have allowed refineries to increase to split of a barrel of crude to more gasoline output and less diesel fuel output.

Now, that's my understanding based on what I have read and my personal observations over the years, but I am quite open to any other opinions of how the concept of diesel fuel as a "waste byproduct" came about...