Low Fret?

I have one sitar sounding fret the high E 2nd fret is buzzing..Do I need to take All the frets down,or can I just do under the high E..They are 6100 frets

I'm assuming it's a telecaster. I would eliminate everything else before filing down frets. I would check the neck relief and action first. Maybe you could just raise the saddle a little bit. Here is a link to Fender's setup guide.

http://www.fender.com/support/articles/telecaster-setup-guide