For some people, concerns about a tax audit or how to maximize tax efficiency pales when laid next to the fact that they can't pay their taxes. Your tax deficit to the government may be the result of any number of circumstances, but the bottom line is that you just may not be able to pay. If you can't pay your taxes, you can try to set up a payment plan with the IRS. But, whatever you do, make sure that you file a return.