That's described in the Code of Federal Regulations 478.11:
Adjudicated as a mental defective. (a) A determination by a court, board, commission, or other lawful authority that a person, as a result of marked subnormal intelligence, or mental illness, incompetency, condition, or disease:
(1) Is a danger to himself or to others; or
(2) Lacks the mental capacity to contract or manage his own affairs.
(b) The term shall include–
(1) A finding of insanity by a court in a criminal case; and
(2) Those persons found incompetent to stand trial or found not guilty by reason of lack of mental responsibility pursuant to articles 50a and 72b of the Uniform Code of Military Justice, 10 U.S.C. 850a, 876b.
As others in the thread have mentioned, some states have mental health procedures that the ATF don't consider to have sufficient due process to constitute a prohibition on the purchase or possession of firearms for federal purposes.