Kevin Costner with some of his kids, including Hayes (right).

Kevin Costner didn’t go easy when it came to directing his son Hayes in his film Horizon: An American Saga.

Speaking with PEOPLE about the new movie, the actor and director, 69, opened up about working with Sienna Miller, who plays his 15-year-old son Hayes’ mother in the film.

After seeing him direct his son, Costner says that Miller had a note for the star.

“She’s our leading lady.

She’s incredible in the movie.

And she came to me one time, she says, ‘You’re kind of hard on Hayes,’ ” Costner tells PEOPLE.

“And I said, ‘I know.

I need him to be really good, and he’s inexperienced and I’m going to help him all the way through.’”

“And in the end, Hayes started doing some things on his own that were just magical, and I’m really proud of his performance,” adds the proud dad.

Kevin Costner with kids.

In addition to Hayes, Costner is dad to six other kids. He shares his three adult kids — Annie, 40, Lily, 37, and Joe, 36 — with his ex-wife Cindy Silva, as well as son Liam, 26, with former partner Bridget Rooney.

He shares kids Cayden Wyatt, 17, Hayes and Grace Avery, 14, with ex-wife Christine Baumgartner.

Earlier this month, Hayes opened up to PEOPLE about what it was like working with his dad on the film.

 

Kevin Costner with his kids.

“It felt like the greatest way that we could bond over something we both love and to create lasting memories together,” said Hayes, who shares his name with Costner’s Horizon character, Hayes Ellison. “Working with my dad was an unforgettable moment in time.”

Hayes said his father was “focused and meticulous” when directing and “always gentle” in offering feedback, but quickly reverted to “dad mode” at the end of the day:

“I think that’s my favorite time with him.”

Over the years, Costner’s other kids have made appearances in his films, including his three oldest who were in his Oscar-winning directorial debut Dances with Wolves in 1990.

“I’m locked in a 70-millimeter film with my children,” said Costner, who recently cast his 36-year-old son Joe in the third chapter of Horizon. “Those were my home movies.”