“It was quite early on I got noticed because I was quite big for my age and I could kick a ball really far! It gave me a massive advantage when playing early on.
“I stood out because of that but I didn't sign for Brighton until I was 12. That's when I started getting proper coaching and taking it seriously.”
As Joel said, Brighton took an interest in him, and it was at the Centre of Excellence (CoE) there that was to be the making of him.
“The CoE at Brighton were brilliant because, not in a bad way, it was quite intimidating, it was quite strict, and they got you in line quite early.
“I didn't feel under pressure, it just gave me discipline. They were only League 2 at the time, but it really made you feel like you were a big deal. I think that's when I realised that's what I wanted to do but also what I had to do to be a professional.
“When I first went into the Centre of Excellence, I don't think I was thinking about it then but the atmosphere gave you an insight into what it means.
“I was still playing for my Sunday team and the school team as well. I remember a point where I think I had a game like every day of the week!
“I was playing for East Sussex Schools, Sussex Schools, Brighton CoE, Ratton Rangers, general school games, so football was literally every day for me until I went full time at Brighton when I was 12.
“There were two centres for the CoE at Brighton, Brighton and Seaford; I was at the Seaford one. There were a few in that group with me, Tommy Elphick, Wes Fogden, John Sullivan then there was Dean Cox and Jake Robinson in the year above me.”
At the age of 12, Joel signed a four-year deal which, at the time, was quite a big thing for someone of that age and the level Brighton were at. By the time that was up at the age of 16, a scholarship was always on the cards for him.
“At 16 I signed my scholarship, and I knew that was what I was going to be doing then and make a real go of it.
“Going into my scholarship, what helped was I'd already played for the reserves as a schoolboy. In that sense I was quite ready for it but going in full-time as a scholar was the real making of me. I thought what was so good was how old school it was, very tough.
“The old school approach, they were very tough on you and you had to work ridiculously hard in order to survive. Real dog eat dog world.
“Dean Wilkins was my youth team coach and we had Martin Hinshelwood who was in charge of the whole youth setup. When I went full-time with Dean, it all came together, he was very strict and very disciplined.
“You got something out of every day and at that age it's a massive change. Coming out of school, which was really easy, to then that real strict full-time environment, was huge. Looking back, Dean was the best thing that happened to me, his coaching is the reason I've had a career.”
Joel progressed quickly as a scholar and was even making the bench as a 16-year-old. From there he eventually made his debut at 18 under Mark McGhee at left-back, and by 19 he was a regular in the side.
It was a bittersweet first season at Brighton, despite being obviously happy to break in the side, Brighton was relegated from the Championship. It was to be the start of the end for him at the Withdean.
“In my second season it all changed, and I played centre-back. Wilkins then became first-team manager which really helped me out having him.
“Dean was sacked then Micky Adams came in and as a young lad I was very confident but naïve as well. It didn't end the best way for me at Brighton or how I'd have liked.
“I was called up for England Under-19s which had a brilliant squad but Brighton wouldn't let me go because they wanted me in a youth cup game as it was the furthest they'd got.
“So, I missed my chance to play for England because I had to play in a youth cup game and I took that badly, I was quite bitter.
“I didn't deal with it in the best way. I still got on with it and thought I might get called up again, but I was young and naïve so took it badly.
“I started getting a bit of interest and had a few offers on the table which Brighton turned down. I wasn't interested in moving at that age really, but it got to a point where I wasn't playing under Micky Adams.
“Nottingham Forest came in for me on loan back up in the Championship. I just thought well I'm not being played in League One so why would I not go and play for a Championship team that will play me.”