Referring to red as conservative and blue as liberal did not occur until the 2000 election. It was brought into service for the voting graphics, of course, to show states, and eventually counties and even smaller units. They provide excellent contrast on the page. It was surprising to me that it caught on in that form, as red had previously referred to communists, associated with the left, not the right. Yet it was in retrospect not surprising. While everyone eventually adopted it, as there was nothing starkly prejudicial about being either of those two colors, as there would have been for yellow, white, black or brown, pink or less-common colors. Good American colors, both of them, right there in the flag. But these were media conventions, and the media was largely leftist, so it bears looking at just a bit more closely.
Assigning red to liberals would never have flown, because they were trying to distance themselves from an association with communists in the public imagination. The upper-class media would protect them on that. But more subtly, look at our other associations with the colors: True blue, blue-blooded, royal blue,and even blue water for sailing, in contrast to redneck, see red, blood red. Even red-blooded, once a term of approval for strength and vitality had become something of a snicker.
If you really want to get down to deep symbolic levels, the colors fit the two Indo-European deities, the sky-father (Zeus, Jupiter, Dyaus Pitar) - light and power, and earth-mother darkness and finally house of the dead. And the sky-father became the more powerful and important one, too. So the blues go into the skies and across the oceans, while the reds stay home on the earth. Thus the assignment of the colors is not fully accidental. No one sat down and said Nyah-ah-ah, we'll fix those conservatives by giving them the worse color. It was just more likely to hold because of the associations. The blues had the media, and the media felt comfortable with that symbolism.
The division into two groups long predates 2000, but it is possibly an indication of the increasing division in our country that this was about the time that we needed to describe the split at that stark a level. Vote Blue, Red State. Stripped down, one-syllable, unnuanced references. We were close to that level of division all the way back to 1976 or further, but third-parties tended to disguise that a bit.
It seems odd to say that I grew up blue, as my mother, my father and stepfather, and all my grandparents were very conservative. But the other relatives I had most contact with were very liberal - uncles, cousins, a predominantly Swedish collection of literary and artistic people. I was also Congregationalist, moving increasingly liberal and eventually going UCC; I was rebelling against said parents and stepparents pretty strongly. But mostly, and it is only in the last few years that this has come clear to me, I was deeply identified with the smart kids, especially of the literary, musical, and theatrical type. Those were my people. And even in a mill city like Manchester, that was a disproportionately Jewish group. Cohen, Klein, Schwartz, Friedman, Goodman - those constituted a full third of my friends. Interestingly, some of the nice Catholic and Orthodox friends I had, I learned years later were half-Jewish. Judaism wasn't quite the guarantee of liberalism that it was in New York or Boston, but it was still very dominant. Those influenced me directly, but were also influencing the popular arts - and "smart kids" culture - nationwide. I have referenced before that much of liberalism comes from picking up social cues, not logical proof. Well, I picked up pretty clearly who the smart kids were and what they believed, and making fun of conservatives was a big part of the culture.
Even going to William & Mary, "the Harvard of the South (heh)" didn't much change that, as W&M was already well on its way to being a Mid-Atlantic rather than Southern school, with one-third of it's student body from the Virginia suburbs of DC, and Pennsylvania, New Jersey, and New York providing much more of the remainder than anywhere in the south, except maybe North Carolina. I had dropped math and science and was in theater, music, and literature. Those were liberal to an average of socialist. They were my people and I took on their surroundings well enough to even be a leader in the small pond way that college provides.
There was residual conservatism in literary culture - even now, the Great Books podcasts I listen to are from National Review. What could be more conservative than studying Latin and Greek? Or Western Civilisation? Or British History? Yet that has been odd, hasn't it? As liberals took over those disciplines they changed them - changed them in focus and method. A great deal of that was to the good, BTW. Why it eventually became toxic is a lengthy discussion for another day. But history, Classics, and all foreign-language studies are as liberal as theater or sociology now. There remains a core of folks studying the Aeneid or Paradise Lost, but that's where the last few conservatives also reside.
The Arts & Humanities, my tribe, became almost entirely Blue. It may be the most influential sector of the group, because of the cultural influence. And you had damn well better fall in line, Jack. The social cues are no longer subtle, but enforceable. You are trapped among them, unable to protest now, and even children's books are not exempt. The annihilation of the past is ongoing.