WALKING through a housing scheme one evening this summer– the sun out, the folk out, bunting strung bright between streetlights – I detected an unfamiliar feeling in the air and in my own heart. It took me a moment to work out what it was: Scottish happiness.

We are not, as a people, prone to joy, and goodness knows there had been little enough to smile about lately, but there was no mistaking it, this sense of shared pleasure, and of course I knew why. It was Fair E’en in Bo’ness, the night before the biggest day in the town calendar.

Folk had beer on their breath, cheer on their minds, as they wandered the streets admiring the Arches: the name given to the huge and extraordinary structures – fairytale castles and the like – built outside the homes of children who would be participating in the following day’s celebrations. Everyone had their glad-rags on. As one Bo’nessian put it: “The hale toon smells o’ new claes.”

I grew up on a council estate very similar to this one, and was familiar with the drab roughcast terraces, but ours were never enlivened by anything like this. The Bo’ness Fair is Brigadoon on acid, the Las Vegas strip made manifest on the Firth of Forth. A giant psychedelic merry-go-round dwarfed one block of houses, and in the middle of a roundabout there was a life-size model of a horse and cart; umbrellas in rainbow colours hung from the trees, sunlight dappling between their spokes.

No one could quite articulate why the Fair was so important. “Better felt than telt” is a phrase you hear a lot round there. To an outsider, myself, it was thrilling, even moving, to perceive the feeling of unity – or, better, kinship – spread through the locals, in the way that a sip of whisky warms the body and stirs the blood. Tomorrow would be Fair Day, a Queen would be crowned from among the town’s children, as she has been since 1897, and the great wheel of the Bo’ness year would have completed another cycle.

I had been visiting the town since January as part of a BBC Scotland crew making a film about the Bo’ness Children’s Fair Festival. It will be part of a new documentary series, Hidden Lives, exploring and celebrating lesser-known aspects of Scottish culture.

Bo’ness seemed a good subject. A pragmatic, no-nonsense sort of place, it is no one’s idea of glamourous. Its identity was forged in the furnace of heavy industry: mining, shipbreaking, the docks. In the 19th and early 20th centuries, Bo’ness was importing so much timber for use as pit props that the town acquired a nick-name, Pitpropolis.

But like a lot of Scottish communities which grew up and cohered around a shared industry now gone, Bo’ness is at risk of losing its distinct identity. What has stopped it from becoming little more than a commuter base for Edinburgh, there can be little doubt, is the town’s deep care for tradition.

Just as those pit props once held up the tunnels, so the Fair holds up Bo’ness. It is these customs and rituals which provide a vital underpinning to Scottish communities, and the communities, taken together, which hold up the nation as a whole.

That Fair Queen, with her crown and sceptre and schoolgirl shyness, is important to Scotland, I think. Queen Elizabeth, when she comes north to Holyrood and Balmoral, does not mean much to the people of Bo’ness. What they value is the wee lassie in their midst because she represents their past, present and future in the way no distant monarch ever could.

The National:

Our film, The Fair Toon, conveys something of this, I hope. It is, in short, a love story.

“The Fair Day is about our community,” Scott McBride, the chairman of the Fair Committee, told me. “From when we pick the Queen in January, the whole town comes together for the six months leading up to the Fair. It’s not just about the one day. And the memories are there forever.”

Another town, another scene, away to the north. Burghead on the Moray Coast is a fishing village on the site of a Pictish fort. It is the evening of January 11, 2019, and a man with a white beard and a bell-shaped fisherman’s hat is carrying a lump of peat across the street. He blows on it as he hurries along, gentling its flame into life. One thinks of snake charmers, horse whisperers, those old shepherds who talk about “kenning” – meaning their understanding of and affinity with the beasts in their charge. This is Dan Ralph, the Clavie King, and what he kens best is fire.

Dan is 70 years old and has been Clavie King since 1988. The title suits him. He is kingly, although perhaps rather priestly, too. He has a face out of Holbein, a voice with the land, sea and sky, the peewit’s cry in it. He is in charge of the Burning Of The Clavie, the fire festival which has been the delight of Burghead for so long that its origins have become lost to time and memory. Is it Pictish? Viking? No one knows for sure.

Hundreds of years ago, when Scotland made the switch from the Julian to the Gregorian calendar, the people of Burghead decided to carry on as they were. They therefore disdain Hogmanay as a new-fangled invention.

Instead, on January 11 they celebrate the old new year by setting on fire a large wooden barrel full of creosote and tarred staves and parading it through crowded streets.

The National:

Dan and the rest of the Clavie crew, an honoured group of strong, courageous and very possibly daft men, take it in turns to carry the burning barrel on their head and shoulders. Just take a moment to consider that. The Clavie is more than six feet tall, weighs a hundred kilos, and – remember – is on fire.

“Och, it’s brilliant fun,” one of the crew told me when I expressed concern for their safety. “In a sense, you’re carrying it for your forefathers, and ensuring that tradition continues.”

Their destination is Doorie Hill, a small steep rise on a promontory overlooking the water on three sides. There, on a round stone altar, the Clavie burns bright and fierce, huge flames shooting high into the air as the crew feed the hungry blaze with buckets of fuel.

It is an extraordinary sight which must be visible, through the darkness, for miles out to sea. And it all begins with that lump of peat carried by the Clavie King. He places it in the barrel, a royal offering, and the ritual is kindled once more.

The Clavie is the name of the first documentary in our series. It was a privilege to be able to film the ceremony and talk to the people who make it happen.

Dan Ralph, in particular, is a fascinating character. The town’s undertaker, he sees Brochers at their lowest, in their grief, and – on Clavie night – at their most joyful. Brocher is what the people of Burghead call themselves.

The village is known as the Broch, which must go back to its ancient origins. This is the only place in Scotland where Pictish stones carved with bulls have been found, and this has become the town’s emblem. That feels appropriate. No better beast could represent Burghead folk – born to be thrawn and refusing the yoke.

The National:

Their stubbornness is an important element in our film. It is the character trait which drives and shapes the Clavie ritual: determination to do things exactly as they have always been done.

This means, for example, using exactly the same boatbuilder’s nail and old pitted ceremonial stone to hammer the barrel on to its pole each year. But it also means that women are not allowed to be members of the crew.

I think it a shame that daughters of Burghead are denied the chance to express their Brocher identity and pride in this most meaningful and public of ways, should they wish to do so. Thinking back to the village’s emblem, one might almost call it macho bull. But I also see the importance of not changing, of making no concessions to modernity; stasis as a means of survival.

Is it just sexism masquerading as traditionalism? Director Fiona Clark and I talked about that a lot, and it is a discussion being had from the Hawick Common Riding to Lerwick’s Up-Helly-Aa. Traditions are fragile things. They need fierce and careful custodianship if they are to endure. And that ferocious care can sometimes rub up against contemporary social attitudes, creating sparks. Our film does not focus on this, but it is an element. Viewers can make up their own minds.

One moment I will always remember from the burning is seeing Annie MacPherson watch the Clavie pass. She was 100 years old, the elder of the village, and though living in care in Elgin, had returned to Burghead for the ritual. The crew stopped outside the house – where she had been born – and handed in a burning stave.

THE Clavie is coated in thick layers of belief and superstition, one of which is that a bit of the burned barrel, kept in the house, will bring good fortune for the coming year. Here, then, was Annie’s piece of luck. On a wild night, it was a tender touch. She was helped, smiling, to the door, and I could see the firelight reflected in her eyes.

It took me back a few years to Up-Helly-Aa, the fire festival in Lerwick, where I met a gentleman called Allan Anderson. He was 76, a widower and retired postman, who, 40 years before, had been the Guizer Jarl: Up-Helly-Aa’s most esteemed figure. Unable now to get down among the crowds, he watched from his bedroom window as the nine-metre dragon-headed Viking galley was borne along the street. The flaming torches of the Jarl squad cast a golden glow on his rapt face.

There is something about the act of witnessing which is important, I think, whether you are a member of the community or outsiders trying, in our case, to capture it on film. Close observation is a form of compassion – to pay intense attention to a person or place or culture, with no more end than trying to tell their story truthfully, there is something loving in that.

I believe that we ought to celebrate our festivals and local customs much more than we do. Newspapers, in particular, should cover these things properly. Not just a picture and a caption, but a proper account. The national press need not be sniffy about giving prominence to stories which they feel are the domain of their local counterparts. There is a way of giving a national treatment to a local story. This stuff is not fluff. Galas, fair days, festivals – these are part of the warp and weft of Scottish society. The Burryman Day, which takes place in South Queensferry each August, is as marvellous and freakily beautiful as anything you might see at the Edinburgh festivals, and it does not deserve to be overshadowed by them.

What I find mystifying is that the Bo’ness Fair, in particular, is not better known. It is spectacular and noisy and heartfelt and funny and bizarre, yet my brother, who has lived only a few miles away for his whole life, had never even heard of it.

The Common Ridings of the Borders get operatic coverage, inky arias devoted to the sight of men on horseback, but Bo’ness remains unsung. I wonder if it is because the Fair is dominated by women and children and is seen on some level as not being serious? If so, that is a great shame. The Bo’ness Fair deserves Scotland’s attention. Not just because it looks extraordinary, but because, just below the surface, it tells a story of the life and health of our towns, which is to say of us.

“Bo’ness isn’t doing too well,” one Fair stalwart told me, meaning that it has problems with lack of jobs and so forth. “It’s not just that the Fair gives the town identity, it’s total escapism, especially for the children. It’s the one day of the year that’s like a fantasy world. A lot of young people live in poverty, but on that day everybody’s the same. You hear kids say that they prefer Fair Day to Christmas.”

I thought of that as I walked past the old winding wheel, on the edge of town, as the sun shone on the water. It is a memorial to the coal industry and is therefore a symbol of absence.

The Fair emerged from the pits, a new seam glittering in the dark; at one time, it was the miners’ only holiday of the year. It was, in other words, a break from the daily struggles of work and money. Now, though, with the heavy industries which shaped Bo’ness and its people gone, it is the Fair which gives the town pride and purpose. The Fair is a laud which contains a lament. One of its most beloved songs, The Best Day Of The Year, makes this explicit:

“We’ve lost our wood yards, pits and docks, our potteries gone forever.

One thing remains and will not die: our Fair – it changes never.”

To hear children sing those words was strange. Did they really understand that mix of sorrow and defiance? Perhaps they will one day. We all come, in the end, to know that song.

For me, it was an honour to hear those kids; an honour to be present at the Fair as a witness, and to feel that elusive Scottish happiness, if only for a moment, one evening, one summer, beneath the big Firth of Forth sky. I hope that we have done justice to these – and other – hidden lives.

Hidden Lives begins on BBC Scotland, October 17 at 8.30pm