The word Bastille is hugely significant for the French - it was an important fortress in Paris whose invasion became a symbol of the French Revolution. But for a band from England, it just sounded cool.
"We were looking for a band name, and my birthday is on July 14, which is Bastille Day. It sounded quite bold and a bit evocative, but it really doesn't mean anything at the same time," lead singer Dan Smith says before quickly adding, "to people in England."
Smith was writing and recording songs by himself before forming a four-man group at the end of 2010. He found drummer Chris Wood from a flyer offering drum lessons, and was introduced to bassist Will Farquarson and keyboardist Kyle Simmons by mutual friends.
"Kyle was a friend of a friend who I always see at parties. Just before we ... started the band, I went through a really weird phase where, if I ever met someone who played an instrument, I would tell them 'You got to join the band!' I must've asked about 50 people." Simmons adds, "And I stuck."
The quartet put a couple of songs online and began playing gigs in the spring of 2011, going on "tour" in just one car. They sometimes even borrowed a car from the mother of a friend, which could fit more of their equipment. Very rock 'n' roll.
The self-proclaimed "baby band" has been picking up steam as of late, with their debut studio album Bad Blood set for release next year, and their first gig outside of Britain a couple of weeks ago at Hong Kong's very own Clockenflap.
"On our last couple of tours, we've been playing in the kind of venues where we, a couple of years ago, were going to see bands we thought were huge. And not that we're huge by any means. But it's just quite weird to be now playing on those stages," says Smith.
As well as the album, Bastille has released two hip-hop-style mixtapes; the latest, Other People's Heartache Pt. 2, came out this month.
The mixtapes allowed the band to experiment without the pressure of keeping "their" sound. Smith has always been obsessed with film, and incorporates elements into his music making.
"I think a lot of the songs are quite self-contained," he says. "They'll have their own little narrative, or sometimes when I'm writing them, I'll almost see each song as a scene in a film."
The upcoming Bad Blood is more representative of them as a band. There is a wide range of sounds on it - electronic, strings, folky and indie to name but a few.
"If we're lucky enough to make a second or third album, I'd like not to have to sit within a genre," Smith says. "It'll be nice to be able to ... do something quite weird in one direction or another."
As for the future, they know better than to rely on a cool name.
"As soon as you start to be known for your music, no one really cares what the band is named," Smith says. "Some of the best bands have the worst names. The Beatles is a terrible name."