NFL Star Jim Brown's View From the Hollywood Hills

My mother, Theresa, had me when she was about 15. A year later, she moved north to Manhasset, N.Y., to work as a live-in domestic.

I saw my father, Swinton, only four times in my life. He was a boxer and dropped by a couple of times when I was little. He had gone off to a city to start another family.

Nora owned our house. It was a one-story weather-beaten home close to the highway that connected the island to the mainland.

Our home was comfortable. It had been in the family since the 1800s. Slave ships from Africa used to land up the coast at Fort Frederica. Nora always used to talk about that.

Nora was a strong woman and very determined. She always wanted me to come in early from playing. If I didn’t, she’d try to catch me but never could. If I slept late, I’d wake up with a switch applied to me.

I learned how to swim in a pond in the middle of the island. We’d put a box in the water and dive off it. The water came up to our necks, so you could stand if you had to. Little by little I figured out how to swim.

My grandmother liked to drink and started early each day. She was drunk nearly all the time and was looked down upon in disgrace. To her credit, she eventually stopped drinking and became a beautiful human being.

The island supplied us with great fish and seafood. We’d catch a lot of our dinners. My favorite meal was Nora’s deviled crabs. She’d remove the meat from the shell and season it spicy.

In 1944, when I was 8, my mother came down to St. Simons and took me up to Manhasset. We lived in an apartment over the garage at the home of the Brockmans, where she worked.

I respected my mother and her determination to send me to school, which was 5 miles away. Sometimes she’d put me in a taxi. She had to work hard to take care of us.

The Brockmans were wealthy and treated us nice. But I was isolated with my mother. There were no other kids to play with. I invented games I could play alone, and I discovered myself.

When my mother changed jobs, we moved to nearby Great Neck, which was closer to the Manhasset schools. First, we lived with the Butlers, an African-American family. This arrangement gave my mother more freedom to be a young woman and make a living. But I didn’t like it. The Butlers had two kids, and I never felt like I fit in. Bill, their son, and I constantly competed for everything.

Several years later my mother and I moved across the street. We lived in the downstairs section of a duplex. The house had a backyard. I dug a pit and set in two poles with a cross bar. That’s where I learned to high-jump.

Mom and I had a difficult relationship. She was a young lady, but I didn’t like her dating men when we lived together. I felt it wasn’t good for her reputation. I even threatened a couple of her boyfriends.

In junior high school, I excelled at lacrosse, basketball, baseball, football and track. I was on a journey. I had only two choices—to be a decent person or a bad person. The discipline of athletics meant I had to play life straight. I had to do the right thing and not indulge in things that would be a detriment to my sports.

In high school, Ed Walsh, my football and basketball coach, became a father figure to me. He was a gentle, caring person who encouraged and guided me. When I met Ed, he knew my potential. Not just as an athlete but as a young man. I trusted him and truly cared about him.

At the end of high school in 1953, I wound up with full scholarship bids to 42 colleges. But Kenny Molloy, a Manhasset lawyer who was involved with my lacrosse team, insisted I attend Syracuse University, his alma mater.

At first, Syracuse didn’t want me, but Kenny made a deal with them. I’d accept a football scholarship and they’d give me a trial period. Kenny paid for my first semester.

I passed the trial, and by senior year I set several college football records. I was picked in the first round of the 1957 NFL draft by the Cleveland Browns. I retired from football in 1965 to become a movie actor.

Today, I live with my wife, Monique, in the Hollywood Hills. I moved here in 1968. It’s a big house that’s perched on the finger of a ridge overlooking the city. We have an amazing, unobstructed view.

My favorite spot is sitting on the deck, facing the house instead of the view. I like to be set should anyone ring the doorbell.

My wife and I also have a condo in Miami’s South Beach, and we still own the house on St. Simons. We’re not sure yet what we’re going to do with it.

When I became a well-known athlete with the Browns, my father reached out to me. I didn’t want to have anything to do with him. I wasn’t angry, just determined not to get caught up in his junk.

I wasn’t going to chase after him my entire life. To me that was weakness. I know who good people are. They helped me and I’m grateful to them. Many were white and wealthy, and they used their resources to put me on the right track.

As for my father, I’m not sorry I didn’t spend time with him. He made his decision, and so did I.