A FEW months ago Bob Dylan celebrated his 78th birthday. For anyone who has followed his career from its efflorescence in the 1960s that should come as no revelation, but it surely does to those of us who heard Forever Young when it was first released in 1974 and were deluded into believing that would be our destiny.

Recently, browsing in a charity shop in Morningside, I found of a copy of a three-CD album called Chimes Of Freedom, released in 2012 to mark 50 years of Amnesty International. It comprised the same number of artists, each of whom had chosen to perform one song by Dylan, including a wonderful version of Forever Young by the late Pete Seeger, who was then in his nineties.

Of course, Dylan did not intend his song to be interpreted, like JM Barrie’s Peter Pan, as a paean to eternal youth. Rather, it is an anthem of defiance and intent, of a challenge to each of us – and to himself – to resist the revisionism and pessimism, cynicism and conservatism that too often accompanies the accumulation of years: “May your heart always be joyful/May your song always be sung.”

As I grow older I find myself listening to Dylan more and more and am mystified when I meet people who have yet “to get into” him or who bang on about his supposed inability to sing.

For me and countless other Dylanophiles, he is simply one of the greatest artists of the 20th century, like Louis Armstrong and Billie Holiday.

He is also the soundtrack to our lives. The first album of his I bought was John Wesley Harding. It was 1968 and the times most certainly appeared to be a-changin’, and not necessarily in a good way. I was 16 and all I need to take me back nearly half a century is to hear the first bars of All Along The Watchtower and I’ll Be Your Baby Tonight. If ever I’m washed up on a desert island they’ll be on my playlist.

Over the years I have had the opportunity to meet people who knew Dylan well – or as well as you can know him – and I tried their patience by interrogating them about him.

Joan Baez was tolerance personified, telling me how on one occasion, in the days when she was more famous than him, they had tried to book in as a couple to a swanky hotel. The manager took one look at a dishevelled Dylan and said he was not the type of person he could possibly have as a guest. Baez was livid but Dylan just shrugged his shoulders and went off alone into the night to find alternative accommodation. The next morning he told her he hadn’t slept a wink, not because he was angry at the snub but because he’d channelled his rejection into writing a song, one of the good ones.

I also interviewed Kris Kristofferson when he was staying with his family for a while in Glasgow. In 1966 he’d got a job in a Nashville recording studio where Dylan and his entourage were working on Blonde On Blonde. For much of the time, recalled Kristofferson, the musicians sat around twiddling their thumbs and drinking beer while waiting for Dylan, who was elsewhere, to finish writing songs such as Just Like A Woman, Rainy Day Women and Visions Of Johanna.

Orchestrating the band was the keyboard player Al Kooper, one of his jobs being to relay Dylan’s instructions. When eventually Dylan appeared the session commenced, almost alchemically. After a few takes, he was happy to move on to the next number. He was not looking for perfection but a sound that defined a song’s essence. Rainy Day Women, for example, was to be performed as if by a Salvation Army Band. Fly-on-the-wall Kristofferson eavesdropped in wonder. “It was like listening to Keats’s words being set to music by Mozart,” he said.

Even Dylan, you would like to think, would approve of that. Or would he? Re-reading Chronicles, the first volume of his memoirs, in preparation for an event this week at the Edinburgh International Book Festival, I was struck by his resistance to cover versions of his songs, few of which he seemed to approve of.

The Byrds, he wrote in Chronicles, had recorded Mr Tambourine Man, but he gave no indication as to what he thought of it, beyond saying “it made it to the top of the charts”, which is not in itself a wholehearted endorsement.

Dylan had begun by singing songs written by others, of course. As an ingenue in Manhattan he was part of the folk scene and well versed in Irish and Scottish ballads as well as “Civil War songs, cowboy songs, songs of lament, church-house songs, anti-Jim Crow songs, union songs...” But unlike other performers who were content to cover others’ original work, Dylan determined to write his own songs, reasoning: “Sometimes you just want to do things your own way, want to see for yourself what lies behind the misty curtain. It’s not like you see songs approaching and invite them in. It’s not that easy. You want to write songs that are bigger than life. You want to say something about strange things that have happened to you, strange things you have seen.”

Anyone interested in Dylan must acknowledge that he reserves the right to remain an enigma. For such a public figure he has retained his unknowability.

In Rolling Thunder Revue, Martin Scorsese’s must-see documentary on Netflix, which follows Dylan on a tour of the US in the mid-1970s, he toys with viewers, appearing at once accessible and distant, approachable and otherworldly. He is the initiator, the puppeteer, the ring-master, and others must fall into line or go elsewhere.

In Chronicles he purports to set the record straight, but as ever with him, one always feels he is never fully graspable. Sure, we learn about his upbringing and his Turkish antecedents, but this portrait of an artist is as remarkable for what it doesn’t disclose as what it does.

We learn, for example, that he was born in 1941 in Hibbing, Minnesota; that his father was “stricken with polio”; and that he grew up in the 1950s when the threat of nuclear armageddon was imminent. “Living under a cloud of fear like this robs a child of his spirit,” he wrote. “It’s one thing to be afraid when someone’s holding a shotgun on you, but it’s another thing to be afraid of something that’s just not quite real. There were lots of folks around who took this threat seriously, though, and it rubbed off on you. It was easy to become a victim of their strange fantasy.”

Like all great storytellers, Dylan believes in the elasticity of facts. Everything must bend to fit his narrative. Reading Chronicles, which was one of the reasons why he won the Nobel Prize for Literature, you are aware that he is being less than forthcoming.

Family details, for instance, are skimpy. We’re told that his father, Abram Zimmerman – who owned an electrical shop – and his mother Beatrice were part of a small Jewish community. No mention is made of his brother, David, his sole sibling.

His formative years are passed over in a few paragraphs. He played in high school bands before moving to Minneapolis where he enrolled at university but did not graduate.

The 1960s were around the corner and rock and roll – in the form of Little Richard and Elvis Presley – was beginning to supplant the crooners who dominated the previous two decades. New York, the epicentre of the music business, was to Dylan what Moscow was to Chekhov’s three sisters. He arrived in the dead of a brutal winter and felt immediately at home. “I could transcend the limitations. It wasn’t money or love that I was looking for. I had a heightened sense of awareness, was set in my ways, impractical and a visionary to boot. My mind was strong like a trap and I didn’t need any guarantee of validity.”

It was in this “freezing metropolis”, in bohemian Greenwich Village, where anything went and one’s past was not so much a foreign country as terra incognita, that Robert Allen Zimmerman metamorphosed into Bob Dylan. The change of names was imperative, ending one chapter of his life and opening another.

Originally, he recalled, he thought about calling himself Robert Allen: “As far as I was concerned, that was who I was – that’s what my parents named me. It sounded like the name of a Scottish king and I liked it. There was little of my identity that wasn’t in it.” Then he happened on some poems by Dylan Thomas and appropriated the Welshman’s first name for his surname. All he need do now was shorten Robert to Bob and he was ready to go. How easily one identity is replaced by another.

Chronicles is Dylan’s Bildungsroman, the story of how he transformed and reinvented himself, how he became an artist who never looks back. To become this person he had first to understand himself, to give him a creed to which he could hold like an anchor through the storm of life. As he wrote in Forever Young, by way of a reminder:

May you have a strong foundation

When the winds of changes shift

He might sing what would glibly be described as protest songs but he was most emphatically not a protest singer. Whenever attempts were made to pigeonhole him as such, he cavilled or did the opposite of what even – especially? – his fans expected.

Thus his going electric or overtly religious or, lately, his embracing of songs more likely to be heard in the repertoire of Bing Crosby or Tony Bennett than the mouthpiece of a so-called generation of rebels.

ONCE, receiving an honorary degree – not at St Andrews, where he was a recipient of one in 2004 – he was described as “the authentic expression of the disturbed and concerned conscience of Young America”. It was all he could to remain expressionless. “Oh Sweet Jesus! It was like a jolt...The disturbed conscience of Young America! There it was again. I couldn’t believe it! Tricked once more.”

For Dylan, the solution to fame was escape. Hence the Rolling Thunder Revue, the Never Ending Tour and the countless nights in anonymous hotels in teeming cities.

If he is on the move he is harder to catch, it is harder to pin on him preposterous epithets – “Prophet, Messiah, Saviour”. For such an extraordinary individual, Dylan craves normalcy and ordinariness, the nine-to-five humdrum of the duty bound. Or so he has sometimes fantasised. In truth, he is like the lonesome hobo of his song of that title with – to invoke the lyric of another of his songs – no destination home.

He has come to recognise that the itinerant existence of life on the road is what makes him what he is. He dare not stand still. He must not mark time. Retreat is the one command he cannot obey.

As he says in Chronicles: “You can write a song anywhere, in a railroad compartment, on a boat, on horseback – it helps to be moving. Sometimes people who have a great talent for writing songs never write any because they are not moving.”

Alan Taylor’s Reading Workshop on Bob Dylan’s Chronicles is at the Edinburgh International Book Festival on Friday, August 16, 11.00am-12.30pm