No this is still OK.
Even though it *looks* like a concatenation, because we will parse the bind first, we'll be OK.
For example, in SQL Plus
--
-- using a bind
--
SQL> variable x varchar2(300);
SQL> exec :x := '''x'' union all select username from dba_users'
PL/SQL procedure successfully completed.
SQL> select * from dual where dummy like '%'||:x||'%';
no rows selected
--
-- compare that to using a substitution
--
SQL> define y = "'x' union all select username from dba_users"
SQL> select * from dual where dummy like '%'||&&y;
old 1: select * from dual where dummy like '%'||&&y
new 1: select * from dual where dummy like '%'||'x' union all select username from dba_users
DUMMY
-----------------------------------------------------------------------------------------------
SYS
SYSTEM
XS$NULL
LBACSYS
OUTLN
DBSNMP
...
But its pleasing to see it raising alarm bells with you. That is probably the most important - to always be on the lookout for risks