Depends on what your harp goals are. I think music for gigs should be memorized because music stands detract from the picture. My rep is all memorized, and I play a free left hand.
When I play opera or orchestra parts, then I have to play what’s really there, and I have to work on it quite a while…I get my parts way ahead of time….once I get the notes down, I practice with the recording. No sight reading here!
Everyone learns differently, and I think harp players are reeeally different from each other. I think your memory ability will serve you well. BTW, I started harp in college, and I could read only the treble clef when I started.
When I got out on my own, I found that written-out music was almost impossible for me to learn, so I just write down the melody (with chord symbols) in a comfortable key, The left hand takes care of itself…and sounds much fuller than in written arrangements.