I just wanted to know how much pga maximum can use system memory ?

It varies by platform and version, but typically you're looking at 2G or 4G for a single session as a ceiling, but several parameters will potentially shrink that down to a far lower size, because the common parameter (pga_aggreate_target) is what controls the allocation of entire pga consumption among sessions. pga_aggregate_limit is a new 12c parameter which sets a hard limit on the pga at the instance level.

You can see an event to dump out additional details whenever you get an ora-4030

alter system set events '4030 trace name errorstack level 3';

but rather than wait for the errors to occur, you can look at v$sesstat for statistics "session pga memory" and "session pga memory max" and keep an eye on what each session is consuming.

iam assuming that most of time, it is caused by sql statement

Unlikely, because we'll start using temporary segment space rather than run out of memory. Common candidates are plsql memory structures growing out of control (associative arrays etc)