Australia was superb throughout and led 2-0 before the 40-minute mark thanks to goals from Lisa De Vanna and Michelle Heyman.
Yuki Ogimi pulled one back for the host nation before the break and there were a few nervy moments in the second half before Katrina Gorry’s late header iced the result.
It’s a huge result against the tournament favourites for Alen Stajcic’s side, who must finish in the top two of the six nation event to reach the Rio Olympics.
After a reasonably even start to the contest, the Westfield Matildas went in front on 25 minutes thanks to a brilliant header by De Vanna.
The experienced co-captain out-muscled her defender and rose highest to plant Katrina Gorry’s excellent cross into the top corner.
It was De Vanna’s 38th goal for the Westfield Matildas, taking her just three behind all-time leading scorer Kate Gill.
De Vanna then turned provider – with a little help from the referee - for Australia’s second to Heyman five minutes before the break.
The 31-year-old ran at the heart of Japan’s defence before slotting a pin-point through-ball for Heyman, who rounded the keeper before rolling it into an empty net.
Japan hit back in first-half stoppage time when Ogimi poked home from close range after intricate build-up play in the box.
Buoyed by that goal, Japan upped the pressure after the break and only some Lydia Williams brilliance denied them an immediate equaliser.
The most impressive of a handful of saves Williams was forced to make came just after the hour mark, diving full-stretch to her left to tip Miyama’s long-range strike around the post.
Stajcic brought on Caitlin Foord and Kyah Simon to try and regain the initiative and it paid off 12 minutes from time.
Emily van Egmond provided the excellent cross to the far post where Gorry ghosted in to head it home and restore Australia’s two-goal lead.
The Aussies then saw out the game with ease to claim a vital three points ahead of Wednesday’s second game against Vietnam.
Westfield Matildas 3 (De Vanna 25’, Heyman 41’, Gorry 78’)
Japan 1 (Ogimi 45+2’)
Crowd: 4,988 @ Kincho Stadium, Osaka
Referee: Carina Vitulano (Italy)
Westfield Matildas starting line-up: 1.Lydia WILLIAMS (gk), 4.Clare POLKINGHORNE (c), 5.Laura ALLEWAY, 6.Chloe LOGARZO (9.Caitlin FOORD 65’), 7.Steph CATLEY, 8.Elise KELLOND-KNIGHT, 10.Emily VAN EGMOND, 11.Lisa DE VANNA (15.Emily GIELNIK 87’), 14.Alanna KENNEDY, 16.Michelle HEYMAN (17.Kyah SIMON 70’), 19.Katrina GORRY
Substitutes not used: 2.Ashleigh SYKES, 3.Caitlin COOPER, 12.Casey DUMONT (gk), 13.Tameka BUTT, 18.Mackenzie ARNOLD (gk), 20.Ellie CARPENTER
Yellow cards: Foord 90+3’
Red cards: Nil
Japan starting line-up: 12.Erina YAMANE (gk), 3.Azusa IWASHIMIZU, 4.Saki KUMAGAI, 5.Aya SAMESHIMA (16.Mana IWABUCHI 84’), 6.Mizuho SAKAGUCHI, 8.Aya MIYAMA (c), 9.Nahomi KAWASUMI (7.Yuri KAWAMURA 84’), 10.Yuki OGIMI, 11.Shinobu OHNO (20.Kumi YOKOYAMA 40’), 13.Saori ARIYOSHI, 14.Emi NAKAJIMA
Substitutes not used: 1.Miho FUKUMOTO (gk), 2.Yukari KINGA, 15.Megumi TAKASE, 17.Asuna TANAKA, 18.Ayaka YAMASHITA (gk), 19.Megumi KAMIONOBE
Yellow cards: Nil
Red cards: Nil