The Gunners had won the first leg 3-0 a week ago and appeared to be complacent in the early exchanges at the Emirates.
Hosam Aiesh and Ken Sema scored within two minutes, as the visitors stunned Arsene Wenger’s men in the first half.
The hosts rallied back in the second half, with Sead Kolasinac hammering in what proved to be a consolation.
Danny Welbeck missed several good opportunities to level the tie for Arsenal, as they qualified 4-2 on aggregate.
Next up for Wenger, is the Carabao Cup final on Saturday against Manchester City.